[issue33524] non-ascii characters in headers causes TypeError on email.policy.Policy.fold

2018-05-15 Thread radical164

New submission from radical164 :

Python 3.6.5 has trouble on folding email messages:

Traceback (most recent call last):
  File "emailtest.py", line 7, in 
policy.fold("Subject", msg["Subject"])
  File "/usr/lib/python3.6/email/policy.py", line 183, in fold
return self._fold(name, value, refold_binary=True)
  File "/usr/lib/python3.6/email/policy.py", line 205, in _fold
return value.fold(policy=self)
  File "/usr/lib/python3.6/email/headerregistry.py", line 258, in fold
return header.fold(policy=policy)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 144, in fold
return _refold_parse_tree(self, policy=policy)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 2651, in 
_refold_parse_tree
part.ew_combine_allowed, charset)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 2728, in 
_fold_as_ew
first_part = to_encode[:text_space]
TypeError: slice indices must be integers or None or have an __index__ method


The message has non-ascii characters in header and set max_line_length=0, 
regardless length of the header. Here is the code to reproduce.


from email.message import EmailMessage
from email.policy import default

policy = default.clone(max_line_length=0)
msg = EmailMessage()
msg["Subject"] = "á"
policy.fold("Subject", msg["Subject"])


I first found this issue on Maildir.add, which saves the message to a file 
without word wrap.

--
components: email
messages: 316680
nosy: barry, r.david.murray, radical164
priority: normal
severity: normal
status: open
title: non-ascii characters in headers causes TypeError on 
email.policy.Policy.fold
type: behavior
versions: Python 3.6

___
Python tracker 
<https://bugs.python.org/issue33524>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33529] Infinite loop on folding email if headers has no spaces

2018-05-15 Thread radical164

New submission from radical164 :

I just reported a bug about email folding at issue 33524, but this issue is 
more fatal in some languages like Chinese or Japanese, which does not insert 
spaces between each words.
Python 3.6.5 has this issue, while 3.6.4 does not.

Create an email with longer header than max_line_length set by its policy.  And 
the header contains non-ascii characters but no white spaces.
When try to fold it, python gets stuck and finally system hangs. There are no 
output unless I stop it with Ctrl-C.

^CTraceback (most recent call last):
  File "emailtest.py", line 7, in 
policy.fold("Subject", msg["Subject"])
  File "/usr/lib/python3.6/email/policy.py", line 183, in fold
return self._fold(name, value, refold_binary=True)
  File "/usr/lib/python3.6/email/policy.py", line 205, in _fold
return value.fold(policy=self)
  File "/usr/lib/python3.6/email/headerregistry.py", line 258, in fold
return header.fold(policy=policy)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 144, in fold
return _refold_parse_tree(self, policy=policy)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 2651, in 
_refold_parse_tree
part.ew_combine_allowed, charset)
  File "/usr/lib/python3.6/email/_header_value_parser.py", line 2735, in 
_fold_as_ew
ew = _ew.encode(first_part)
  File "/usr/lib/python3.6/email/_encoded_words.py", line 215, in encode
blen = _cte_encode_length['b'](bstring)
  File "/usr/lib/python3.6/email/_encoded_words.py", line 130, in len_b
groups_of_3, leftover = divmod(len(bstring), 3)
KeyboardInterrupt


Code to reproduce:

from email.message import EmailMessage
from email.policy import default

policy = default # max_line_length = 78
msg = EmailMessage()
msg["Subject"] = "á"*100
policy.fold("Subject", msg["Subject"])


No problems in following cases:

1. If the header is shorter than max_line_length.
2. If the header can be split with spaces and the all chunk is shorter than 
max_line_length.
3. If the header is fully composed with ascii characters. In this case, there 
is no problem even if it is very long without spaces.

--
components: email
messages: 316747
nosy: barry, r.david.murray, radical164
priority: normal
severity: normal
status: open
title: Infinite loop on folding email if headers has no spaces
type: behavior
versions: Python 3.6

___
Python tracker 
<https://bugs.python.org/issue33529>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com