Re: Proposal to upgrade django.core.mail to Python's modern email API

[Mike Edmunds]

On Friday, June 28, 2024 at 1:07:35 PM UTC-7 Florian Apolloner wrote:
> Are all of those documented? If not we can simply remove them (especially 
if the deprecation implementation turns out to be a PITA).

It sort of depends on the definition of "documented." I've dug into 
real-world usage; results and proposals below.

Incidentally, I thought there was (used to be?) a policy that internal 
undocumented APIs were fair game for use by third-party libraries, 
subclassing, etc., so long as they didn't start with an underscore. (But 
"private" underscore APIs could have breaking changes at any time.) Am I 
remembering that wrong? Or was internal API stability only guaranteed for 
patch-level releases?

*Should go through deprecation process* (in my opinion, sorted roughly by 
real-world usage)

   - *BadHeaderError* is mentioned in the docs 
   

 as 
   potentially raised during sending
  - Exposed in django.core.mail.__all__
  - Real-world use: ~2300+ cases in GitHub code search 
  
,
 
  seems to be primarily error handling 
  

  - Proposal: deprecate on import. (Modern Python email API raises 
  ValueError for newlines in headers. In our deprecation, we'll need 
  `BadHeaderError = ValueError`—rather than subclassing ValueError—to 
  preserve behavior of existing error handling code.)
  - Suggested replacement for docs: change `except BadHeaderError` to 
  `except ValueError`
  
  - *sanitize_address()* is not documented, but is widely used
  - Not exposed in django.core.mail (have to import from 
  django.core.mail.message)
  - Real-world use: ~430 cases in GitHub code search 
  
,
 
  often in custom EmailBackend implementations 
  

 or 
  for special case handling 
  

 that 
  needs knowledge of how django.core.mail handles addresses.
  - Proposal: deprecate. (Have to keep implementation around anyway as 
  part of other deprecated code.)
  - Suggested replacement for docs: none (it's an internal, 
  undocumented API). Or we could say it depends on the use case: just skip 
it 
  if no longer relevant, use something like Python's modern email Address 
  object if you need formatting cleanup, or copy sanitize_address() into 
your 
  code and adapt for your needs. (Incidentally, it's already pretty 
  common to copy 
  

 
  and adapt sanitize_address().)
  
  - *SafeMIMEText* and *SafeMIMEMultipart* are mentioned in the docs 
   

 
   as the return type of EmailMessage.message(), but not further documented
  - Exposed in django.core.mail.__all__
  - Real-world use: ~240 cases in GitHub code search 
  
,
 
  both for type checking 
  

 
  and for constructing text or message attachments 
  

  - Proposal: deprecate (on import, to catch type checking uses)
  - Suggested replacement for docs: django.mail.EmailMessage.message() 
  will now return an email.message.EmailMessage; to construct text or 
message 
  attachments, switch to modern email.message.EmailMessage
  - [Tip for django-stubs: SafeMIMEText, SafeMIMEMultipart, and 
  email.message.EmailMessage are all subclasses of email.message.Message]
  
  - *forbid_multi_line_headers()* is not documented, but seems to be an 
   intentional part of the public API for implementin

Re: Proposal to upgrade django.core.mail to Python's modern email API

[Mike Edmunds]

About MIMEBase attachments: I propose we continue supporting them in 
Django, without deprecation, for now. We can investigate adding support for 
the equivalent in Python's modern email API later, as a separate proposal.

More details…

On Wednesday, June 26, 2024 at 6:28:06 PM UTC-7 I wrote:
> Tricky bits and things requiring longer discussion…
> *Legacy MIMEBase attachments:* I'll post a separate message about this 
sometime later.

Since 2007 
,
 
Django has supported and documented 

 
using MIMEBase objects in Django's EmailMessage.attachments list. 

Using MIMEBase is necessary for any "complex attachment" that can't be 
simply expressed as filename + content data + mimetype. This includes 
things like text attachments with a different charset than the main 
message, inline images (which require Content-Disposition: inline and 
Content-ID MIME headers), adding params to the Content-Type header, or in 
general anything where you need additional control over the attachment's 
MIME headers and content encoding.

MIMEBase is part of Python's legacy 
 email 
API. The modern email replacement is add_attachment() 
,
 which 
offers everything we need for simple attachments. It also supports complex 
attachments via "contentmanager" kwargs 

 
like `disposition` for the Content-Disposition header, `cid` for 
Content-ID, and `params` for Content-Type extensions. But add_attachment() 
doesn't support MIMEBase.

So as part of this proposal, we'll handle simple filename+content+mimetype 
attachments through modern add_attachment(). I had originally planned to 
treat MIMEBase attachments as deprecated, and find a way to convert them to 
modern add_attachment() kwargs. And I was going to propose a way to somehow 
include add_attachment() kwargs directly in Django's 
EmailMessage.attachments, as the non-deprecated way to specify complex 
attachments. I still think this is the right long-term direction.

But in the interests of limiting scope for the current proposal, I think we 
can treat that as a separate project, and just continue to support MIMEBase 
attachments for now:

   - From what I can tell, it's still possible to attach legacy MIMEBase 
   objects to a modern Python EmailMessage, and get results that work at least 
   as well as they did with entirely legacy APIs.
   - You can also opt into the modern API when creating a MIMEBase object, 
   by passing policy=email.policy.default to its constructor. This might avoid 
   some obscure bugs that occur with the legacy APIs. I think Django's docs 
   should suggest this approach "for improved compatibility" or something like 
   that.
   - When the time comes to support complex Django EmailMessage.attachments 
   using Python's modern add_attachment() kwargs, I think we should also look 
   into supporting Python EmailMessage's add_related() 
   
.
 
   Right now, there's no way to get Django to send a properly constructed 
   

 
   multipart message with html, inline images, and attachments: the inline 
   images end up in the wrong part. Most email clients aren't that picky, and 
   display the inlines anyway (if they handle inline images at all), but it 
   would be helpful to have a way to generate the correct message structure. 
   Just… later.


- Mike

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9aa610dd-01de-4f98-bcfa-f982872ebdban%40googlegroups.com.

Re: Proposal to upgrade django.core.mail to Python's modern email API

[Mike Edmunds]

Thanks to everyone for the feedback so far.

It looks like all the responses to date are generally positive, and I 
haven't seen any objections, so I'm going to open a ticket.

(Of course, additional feedback—positive or negative—and advice is very 
much appreciated.)

Cheers,
Mike

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/092bfdd6-4161-4d5c-a21b-82e778f98ec7n%40googlegroups.com.