Public bug reported: Dear Maintainers,
there's a bug in the Mailman3 package that leads to a corrupted held messages queue. The problem manifests itself by a 500 server error when the list administrator tries to access the respective list's "held messages" panel in Postorius. Expected behaviour is that the held messages queue is shown. Details of the problem can be found in this thread: https://lists.mailman3.org/archives/list/mailman-us...@mailman3.org/thread/QWKXWB3D6GV7CACNH2I7A3JCM7OUHR3G/ Summary: If Mailman is served via Postfix, and Postfix has set the "smtputf8_enable" configuration option set to "yes" (which as of Postfix 3.0.0 is the default value), then e-mail messages with broken Unicode content are changed to include the U+FFFD REPLACEMENT CHARACTER. Such broken messages are usually spam messages from non-members, and thus end up in the held messages queue. There the U+FFFD character hits a bug in Mailman3's e-mail processor, causing it to crash with a UnicodeEncodeError, thus leading to the 500 server error when accessing the held messages queue. Once the corrupt message is in the held messages queue, there's no easy way to remove it from it again. Remedies can be found in the linked mailing list thread. The problem is known upstream and has been fixed: https://gitlab.com/mailman/mailman/issues/441 https://gitlab.com/mailman/mailman/merge_requests/350 It's an underlying problem in Python itself, reported here: https://bugs.python.org/issue32330 For the record, this is the exception found in mailman.log: Dec 09 13:21:04 2019 (3090) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3.6/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 215, in falcon.api.API.__call__ (falcon/api.c:2872) File "falcon/api.py", line 189, in falcon.api.API.__call__ (falcon/api.c:2419) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get resource = self._make_collection(request) File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 78, in _make_resource resource['msg'] = msg.as_string() File "/usr/lib/python3.6/email/message.py", line 158, in as_string g.flatten(self, unixfrom=unixfrom) File "/usr/lib/python3.6/email/generator.py", line 116, in flatten self._write(msg) File "/usr/lib/python3.6/email/generator.py", line 181, in _write self._dispatch(msg) File "/usr/lib/python3.6/email/generator.py", line 214, in _dispatch meth(msg) File "/usr/lib/python3.6/email/generator.py", line 243, in _handle_text msg.set_payload(payload, charset) File "/usr/lib/python3.6/email/message.py", line 315, in set_payload payload = payload.encode(charset.output_charset) UnicodeEncodeError: 'ascii' codec can't encode character '\ufffd' in position 269: ordinal not in range(128) Please backport this fix. A corrupted held messages queue makes the moderation of e-mails impossible. Package mailman3 version: 3.1.1-9 Best Regards, Lauri Ojansivu ** Affects: mailman3 (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1856181 Title: Broken unicode e-mails corrupt mailman's held message queue To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/mailman3/+bug/1856181/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
