Since the early feedback seems positive (though we're still waiting for more votes), here's some additional detail on the changes I think would be involved to update django.core.mail to use Python's modern email API.
(See my earlier message <https://groups.google.com/g/django-developers/c/2zf9GQtjdIk/m/GWjimue9FAAJ> for background on Python's legacy vs. modern email APIs and why updating is useful.) Note: Django and Python both have classes named EmailMessage. I'm using "Django EmailMessage" to refer to django.core.mail.message.EmailMessage, and "Python EmailMessage" (or sometimes just "modern API") to refer to Python's modern email.message.EmailMessage. Necessary work <https://gist.github.com/medmunds/07720063fafef9e6980cc9fba2760986#necessary-work> 1. Update tests.mail.tests to use modern email APIs Where a test relies on legacy implementation details, try to rewrite it to be implementation agnostic if possible; otherwise try to retain the spirit of the test using modern APIs. Retain all security related tests, with updates as appropriate. (Even where we know Python's modern API handles the security for us, it doesn't hurt to double check.) Probably need to add some cases for existing behavior not currently covered by tests, particularly around attachments and alternatives. Remove a test case *only* if it's truly no longer relevant and can't be usefully updated. (And perhaps leave behind a brief comment explaining why.) (Legacy email APIs used in tests.mail.tests: email.charset, email.header.Header, MIMEText, parseaddr. Also message_from_… currently defaults to legacy policy.compat32.) 2. Update django.core.mail.message.EmailMessage to use modern email APIs Change Django's EmailMessage.message() to construct a modern email.message.EmailMessage rather than a SafeMIME object (which is based on legacy email.message.Message with policy=compat32). Add a message(policy=default) param forwarded to Python's EmailMessage constructor. Hoist alternative part handling from Django's EmailMultiAlternatives into Django's base EmailMessage to simplify the code. (More on this below.) In _create_alternatives(), use modern add_alternative() <https://docs.python.org/3/library/email.message.html#email.message.EmailMessage.add_alternative> and friends to replace legacy SafeMIME objects. In _create_attachments(), use modern add_attachment() <https://docs.python.org/3/library/email.message.html#email.message.EmailMessage.add_attachment>. Handle (legacy Python) MIMEBase objects as deprecated, and convert to modern equivalent. This is a relatively complex topic; I'll post a separate message about it later. (I also have some questions about how best to handle content-disposition "inline" and whether we want to somehow support multipart/related.) Remove the private Django EmailMessage methods _create_mime_attachment() and _create_attachment() (without deprecation). (Legacy APIs used in django.core.mail.message: email.message.Message, email.charset, email.encoders, email.header, email.mime, email.utils) 3. Deprecate unused internal APIs from django.core.mail.message Django will no longer need these (Python's modern email API covers their functionality), but they may be in use by third-party libraries: - utf8_charset - utf8_charset_qp - RFC5322_EMAIL_LINE_LENGTH_LIMIT - BadHeaderError^ (more details below) - ADDRESS_HEADERS - forbid_multi_line_headers()^ - sanitize_address() (more details below) - MIMEMixin - SafeMIMEMessage - SafeMIMEText^ - SafeMIMEMultipart^ (Items marked ^ are exposed in django.core.mail via __all__. I haven't looked into the reason for that.) 4. Update django.core.mail.__init__ to avoid EmailMultiAlternatives, and django.core.mail.backend.smtp to replace sanitize_address. (More details below.) 5. Update docs - deprecation of legacy MIMEBase in attachments list, what to do instead - eliminate EmailMessage/EmailMultiAlternatives distinction - deprecation of internal legacy items from #3 Tricky bits and things requiring longer discussion <https://gist.github.com/medmunds/07720063fafef9e6980cc9fba2760986#tricky-bits-and-things-requiring-longer-discussion> - *Legacy MIMEBase attachments:* I'll post a separate message about this sometime later. (Maybe next week. It's a long topic, plus I'm still experimenting.) - *EmailMultiAlternatives:* I'd like to get rid of the distinction between Django's EmailMessage and EmailMultiAlternatives, and just move the logic for alternative parts into base EmailMessage. (And then simplify the docs.) EmailMultiAlternatives is additional complexity for users, and it isn't really helpful with Python's modern add_alternative() <https://docs.python.org/3/library/email.message.html#email.message.EmailMessage.add_alternative> API. We could deprecate EmailMultiAlternatives, but that would be a very noisy deprecation. (Everything sending plaintext + html email today *has* to use EmailMultiAlternatives.) My preference would just be to leave a stub class with a doc string: class EmailMultiAlternatives(EmailMessage): # TODO: add doc string explaining what this used to be, # and suggesting just using EmailMessage instead. pass - *BadAddressHeader:* is a ValueError subclass raised only by forbid_multi_line_headers()—which would be deprecated. (The modern email API raises a ValueError for CR/NL in headers.) Do we want to issue a deprecation warning for this? The warning has to be on import, not on use. (There's a way to do that with a module-level __getattr()__, which I see we've done before in db.models.enums.) Otherwise I'm inclined to just write BadAddressHeader = ValueError in the deprecated code. (forbid_multi_line_headers() will warn about deprecation when it's called.) Same question about all the deprecated constants (like RFC5322_EMAIL_LINE_LENGTH_LIMIT). - *sanitize_address():* This function is called from two places in Django: - forbid_multi_line_headers() (which would be deprecated) - Django's *SMTP EmailBackend* to process the "envelope" from and recipient addresses (which may not be the same as the *From*, *To*, etc. message headers—e.g., bcc is only in envelope recipients, not message headers) sanitize_address() does a bunch of things: - Raises ValueError some forms of invalid addresses - Checks again for CR/NL, and raises ValueError (not BadAddressHeader) - Encodes/re-encodes non-ascii display names to ascii RFC 2047 (using a mix of legacy and modern email APIs that may be the source of some bugs) - Encodes/re-encodes non-ascii addr-specs to ascii RFC 2047 mailbox + punycode domain The modern email APIs handle all of this *for message headers.* (Except the punycode part, which is arguably wrong in message header fields.) However, I believe at least some of this is necessary for smtplib. In particular, the envelope from and recipient addresses have to be ascii only. (Unless SMTP.sendmail() is called with the "SMTPUTF8" option—which is not currently possible with Django's SMTP EmailBackend—and the SMTP server supports that option.) I'd suggest we deprecate sanitize_address(), and create a simpler private function in the SMTP EmailBackend to perform the SMTP-specific ascii conversion. - *Other stuff:* I haven't yet investigated what to do with: - Django EmailMessage.encoding and settings.DEFAULT_CHARSET - Django EmailMultiAlternatives.alternative_subtype (when overridden) -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/0041c2f3-2ba4-4e76-b7df-b3dc025eb4ean%40googlegroups.com.
