** Description changed:
[impact]
python-twisted errors out with "'NoneType' object has no attribute
'encode' in requestReceived()" when it tries to parse a multipart mime
message and python3.7+ is used. This happens because before commit
cc3fa20 in cpython, cgi.parse_multipart ignored parts without a name
defined in "content-disposition" (or parts without headers for that
matter) but after 3.7+ the return of the function can now contain
NoneType keys, which fail to encode.
[scope]
- This bug affects all releases
+ Even though this bug affects all python3-twisted releases, I'll backport
+ the fix just to Focal, Groovy and Hirsute. Bionic and Xenial do not have
+ Python 3.7 as the default interpreter (required to trigger the issue),
+ and the delta in python3-twisted might be to big to backport as the
+ current packages do not contemplate _PY37PLUS at all.
Fixed upstream with commit 310496249, available since 21.2.0rc1
[test case]
1. Save the following code as webserver.py
from twisted.application.internet import TCPServer
from twisted.application.service import Application
from twisted.web.resource import Resource
from twisted.web.server import Site
-
class Foo(Resource):
- def render_POST(self, request):
- newdata = request.content.getvalue()
- print(newdata)
- return ''
-
+ def render_POST(self, request):
+ newdata = request.content.getvalue()
+ print(newdata)
+ return ''
root = Resource()
root.putChild("foo", Foo())
application = Application("cgi.parse_multipart test")
TCPServer(8080, Site(root)).setServiceParent(application)
2. Save the following code as client.py (python3-httplib2 is required)
#!/usr/bin/env python
import httplib2
-
def http_request(url, method, body=None, headers=None, insecure=False):
- """Issue an http request."""
- http = httplib2.Http(disable_ssl_certificate_validation=insecure)
- if isinstance(url, bytes):
- url = url.decode("ascii")
- return http.request(url, method, body=body, headers=headers)
-
+ """Issue an http request."""
+ http = httplib2.Http(disable_ssl_certificate_validation=insecure)
+ if isinstance(url, bytes):
+ url = url.decode("ascii")
+ return http.request(url, method, body=body, headers=headers)
url = "http://localhost:8080";
method = "POST"
headers = {'Content-Type': 'multipart/form-data;
boundary="8825899812428059282"'}
emptyh = '--8825899812428059282\n\n--8825899812428059282--'
print("== BODY: " + emptyh + "\n")
response, content = http_request(url, method, emptyh, headers)
3. Run the server with "twistd3 -y webserver.py"
4. Run the client
5. twistd will fail to encode the key and will drop this traceback in the log
file (twistd.log)
2021-02-16T13:41:39+0100 [_GenericHTTPChannelProtocol,7,127.0.0.1] Unhandled
Error
- Traceback (most recent call last):
- File "/usr/lib/python3/dist-packages/twisted/python/log.py", line
103, in callWithLogger
- return callWithContext({"system": lp}, func, *args, **kw)
- File "/usr/lib/python3/dist-packages/twisted/python/log.py", line
86, in callWithContext
- return context.call({ILogContext: newCtx}, func, *args, **kw)
- File "/usr/lib/python3/dist-packages/twisted/python/context.py",
line 122, in callWithContext
- return self.currentContext().callWithContext(ctx, func, *args,
**kw)
- File "/usr/lib/python3/dist-packages/twisted/python/context.py",
line 85, in callWithContext
- return func(*args,**kw)
- --- <exception caught here> ---
- File
"/usr/lib/python3/dist-packages/twisted/internet/posixbase.py", line 614, in
_doReadOrWrite
- why = selectable.doRead()
- File "/usr/lib/python3/dist-packages/twisted/internet/tcp.py", line
243, in doRead
- return self._dataReceived(data)
- File "/usr/lib/python3/dist-packages/twisted/internet/tcp.py", line
249, in _dataReceived
- rval = self.protocol.dataReceived(data)
- File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2952, in dataReceived
- return self._channel.dataReceived(data)
- File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2245, in dataReceived
- return basic.LineReceiver.dataReceived(self, data)
- File "/usr/lib/python3/dist-packages/twisted/protocols/basic.py",
line 579, in dataReceived
- why = self.rawDataReceived(data)
- File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2252, in rawDataReceived
- self._transferDecoder.dataReceived(data)
- File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
1699, in dataReceived
- finishCallback(data[contentLength:])
- File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2115, in _finishRequestBody
- self.allContentReceived()
- File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2224, in allContentReceived
- req.requestReceived(command, path, version)
- File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
898, in requestReceived
- self.args.update({
- File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
899, in <dictcomp>
- x.encode('iso-8859-1'): \
- builtins.AttributeError: 'NoneType' object has no attribute 'encode'
+ Traceback (most recent call last):
+ File "/usr/lib/python3/dist-packages/twisted/python/log.py", line
103, in callWithLogger
+ return callWithContext({"system": lp}, func, *args, **kw)
+ File "/usr/lib/python3/dist-packages/twisted/python/log.py", line
86, in callWithContext
+ return context.call({ILogContext: newCtx}, func, *args, **kw)
+ File "/usr/lib/python3/dist-packages/twisted/python/context.py",
line 122, in callWithContext
+ return self.currentContext().callWithContext(ctx, func, *args,
**kw)
+ File "/usr/lib/python3/dist-packages/twisted/python/context.py",
line 85, in callWithContext
+ return func(*args,**kw)
+ --- <exception caught here> ---
+ File
"/usr/lib/python3/dist-packages/twisted/internet/posixbase.py", line 614, in
_doReadOrWrite
+ why = selectable.doRead()
+ File "/usr/lib/python3/dist-packages/twisted/internet/tcp.py", line
243, in doRead
+ return self._dataReceived(data)
+ File "/usr/lib/python3/dist-packages/twisted/internet/tcp.py", line
249, in _dataReceived
+ rval = self.protocol.dataReceived(data)
+ File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2952, in dataReceived
+ return self._channel.dataReceived(data)
+ File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2245, in dataReceived
+ return basic.LineReceiver.dataReceived(self, data)
+ File "/usr/lib/python3/dist-packages/twisted/protocols/basic.py",
line 579, in dataReceived
+ why = self.rawDataReceived(data)
+ File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2252, in rawDataReceived
+ self._transferDecoder.dataReceived(data)
+ File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
1699, in dataReceived
+ finishCallback(data[contentLength:])
+ File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2115, in _finishRequestBody
+ self.allContentReceived()
+ File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
2224, in allContentReceived
+ req.requestReceived(command, path, version)
+ File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
898, in requestReceived
+ self.args.update({
+ File "/usr/lib/python3/dist-packages/twisted/web/http.py", line
899, in <dictcomp>
+ x.encode('iso-8859-1'): \
+ builtins.AttributeError: 'NoneType' object has no attribute 'encode'
[regression potential]
This affects the returned dictionaries with non-str keys, which were
discarded in python3.6 or earlier before they reached twisted, so
patching this will make its behavior consistent.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1915819
Title:
'NoneType' object has no attribute 'encode' in requestReceived() when
multipart body doesn't include content-disposition
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/twisted/+bug/1915819/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
