[issue12687] Python 3.2 fails to load protocol 0 pickle

Antoine Pitrou added the comment: Fixed with a test. -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___ ___

[issue12687] Python 3.2 fails to load protocol 0 pickle

Roundup Robot added the comment: New changeset c47bc1349e61 by Antoine Pitrou in branch '3.2': Issue #12687: Fix a possible buffering bug when unpickling text mode (protocol 0, mostly) pickles. http://hg.python.org/cpython/rev/c47bc1349e61 New changeset 6aa822071f4e by Antoine Pitrou in branch

[issue12687] Python 3.2 fails to load protocol 0 pickle

Vinay Sajip added the comment: I confess I'm not familiar enough with the pickle module internals to be sure of putting in the right fix quickly. I will take a look at _Unpickler_Readline when I get a chance, if someone doesn't beat me to it :-) -- ___

[issue12687] Python 3.2 fails to load protocol 0 pickle

Antoine Pitrou added the comment: Ok, the patch is not correct. The core issue is that _Unpickler_Readline should always return a \0-terminated string, but sometimes it doesn't; this issue should be fixed instead of working around it in some other function. -- ___

[issue12687] Python 3.2 fails to load protocol 0 pickle

Vinay Sajip added the comment: I can add a test, using the data attached to the ticket, but like the marshal case we discussed before, it might be several KB of data, which I would incorporate into the tests using a similar approach to the one I used for marshal. (This data has been shrunk fr

[issue12687] Python 3.2 fails to load protocol 0 pickle

Antoine Pitrou added the comment: First, the patch calls PyOS_strtol while a Py_ssize_t should be decodable. However, the dump phase (in memo_put) coerces the memo size to long as well, so this shouldn't be a problem in real life. Second, the patch needs a test. Also, please click on the "re

[issue12687] Python 3.2 fails to load protocol 0 pickle

Changes by Vinay Sajip : Added file: http://bugs.python.org/file22876/fac0421cb7b2.diff ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue12687] Python 3.2 fails to load protocol 0 pickle

Changes by Vinay Sajip : -- keywords: +patch Added file: http://bugs.python.org/file22872/6394321ef4ec.diff ___ Python tracker ___ ___

[issue12687] Python 3.2 fails to load protocol 0 pickle

Changes by Vinay Sajip : -- hgrepos: +58 resolution: -> accepted stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list

[issue12687] Python 3.2 fails to load protocol 0 pickle

Andrew Wilkins added the comment: In _pickle.c, the load_put function calls _Unpickler_Readline, which may prefetch data and place it after the line read in with "readline". load_put then calls PyLong_FromString, which doesn't like the trailing data after the '\n'. Maybe just use PyOS_strtol

[issue12687] Python 3.2 fails to load protocol 0 pickle

Vinay Sajip added the comment: I also noticed that in the file there are numerous previous integer values in a similar context, which were parsed without error. For example, if you look at the test.bin file in an editor, the failure occurs while parsing line 515. Notice the similar constructs

[issue12687] Python 3.2 fails to load protocol 0 pickle

New submission from Vinay Sajip : The attached 2.x-written protocol 0 pickle file cannot be loaded by Python 3.2 or 3.3, though it loads successfully in 2.x. Code used to load: data = pickle.load(open('test.bin', 'rb')) Error: Traceback (most recent call last): File "load_it.py", line 4, i