[issue8383] pickle is unable to encode unicode surrogates

STINNER Victor added the comment: Commited: r80031 (py3k) and r80032 (3.1), fix also pickletools. -- resolution: -> fixed status: open -> closed ___ Python tracker ___ _

[issue8383] pickle is unable to encode unicode surrogates

Marc-Andre Lemburg added the comment: STINNER Victor wrote: > > STINNER Victor added the comment: > >> Both pickle and marshal will need to use the new error handler >> in order to stay compatible with Python 3.0 (and 2.x) >> and also to enable creating Unicode literals that include >> lon

[issue8383] pickle is unable to encode unicode surrogates

STINNER Victor added the comment: > Both pickle and marshal will need to use the new error handler > in order to stay compatible with Python 3.0 (and 2.x) > and also to enable creating Unicode literals that include > lone surrogates. Attached patch fixes pickle. Marshal does already use surr

[issue8383] pickle is unable to encode unicode surrogates

Marc-Andre Lemburg added the comment: Both pickle and marshal will need to use the new error handler in order to stay compatible with Python 3.0 (and 2.x) and also to enable creating Unicode literals that include lone surrogates. -- ___ Python trac

[issue8383] pickle is unable to encode unicode surrogates

STINNER Victor added the comment: I found this bug indirectly: test_logging failed on a SocketHandler if LogRecord.pathname contains a surrogate character. SocketHandler uses pickle to serialize the record. -- ___ Python tracker

[issue8383] pickle is unable to encode unicode surrogates

Changes by STINNER Victor : -- keywords: +patch Added file: http://bugs.python.org/file16904/pickle_surrogates.patch ___ Python tracker ___ ___

[issue8383] pickle is unable to encode unicode surrogates

New submission from STINNER Victor : Python3 uses unicode surrogates to store undecodable filenames. Eg. the filename b"abc\xff.py" is encoded as "abc\xdcff.py" if the file system encoding is ASCII. Pickle is unable to store them: ./python -c 'import pickle; pickle.dumps("abc\udcff")' (...) Un