[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Georg Brandl added the comment: Thanks! -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Roundup Robot added the comment: New changeset 42dd11028e94 by Vinay Sajip in branch 'default': Closes #12291 for 3.3 - merged fix from 3.2. http://hg.python.org/cpython/rev/42dd11028e94 -- ___ Python tracker

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Roundup Robot added the comment: New changeset edba722f3b02 by Vinay Sajip in branch '3.2': Closes #12291: Fixed bug which was found when doing multiple loads from one stream. http://hg.python.org/cpython/rev/edba722f3b02 -- nosy: +python-dev resolution: -> fixed stage: patch review

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Georg Brandl added the comment: Can we get this committed for 3.2.1 then? -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Antoine Pitrou added the comment: Latest patch looks ok to me. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: As a result of the changes to marshal.c, test_importlib needs a small change: the test_bad_marshal raises an EOFError now, whereas it raised ValueError before. I think it's because the earlier code in marshal didn't properly check for EOF conditions in some pla

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip : Added file: http://bugs.python.org/file22530/9993567039c0.diff ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: > Antoine Pitrou added the comment: > It's not proscribed, but trying to remove the "self." because it's > supposed to be more readable is a bit of a strange thing to do. > Also, people reading the test suite should be accustomed to > "self.assertEqual" anywa

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip : Added file: http://bugs.python.org/file22528/c3ba4f7b5db1.diff ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Antoine Pitrou added the comment: > > - "assertEqual = self.assertEqual" is more of a nuisance than anything > > else; > >you're making the code more complicated to read just to save a few > >keystrokes; > >same for "assertIsInstance = self.assertIsInstance" > > I'm not sure how it's mor

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: > Antoine Pitrou added the comment: > > I think the question is: will the slowdown apply to module import, or only > to >explicit uses of the marshal module? If the latter, then I think we can live >with it - we discourage using marshal as a general-purpose

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Antoine Pitrou added the comment: I think the question is: will the slowdown apply to module import, or only to explicit uses of the marshal module? If the latter, then I think we can live with it - we discourage using marshal as a general-purpose serialization scheme anyway. As for the patc

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: > Just a nit, could you give descriptive file names to your patches? > Hex numbers quickly get confusing. Ok - I was under the impression that those names were generated automatically from the changeset hash, and that changing the name arbitrarily would break so

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Dirkjan Ochtman : -- nosy: +djc ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Antoine Pitrou added the comment: Le Sun, 26 Jun 2011 19:49:03 +, Vinay Sajip a écrit : > > Added file: http://bugs.python.org/file22487/0feab4e7b27f.diff Just a nit, could you give descriptive file names to your patches? Hex numbers quickly get confusing. -- ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: I've incorporated Engelbert's two patches into test_marshal.py in my sandbox fix branch. I coalesced them into a single additional test in the existing BugsTestCase. -- type: -> behavior ___ Python tracker

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip : Added file: http://bugs.python.org/file22487/0feab4e7b27f.diff ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

engelbert gruber added the comment: add interleaved writing to the same file (which might happen or not?) the tests pass when vinay's marshal.c is applied. sorry for another patch -- Added file: http://bugs.python.org/file22479/multiple_dump_load_read_write_test.patch __

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

engelbert gruber added the comment: patch to test_marshal.py that obviously fails in current implementation. IMHO is file might not be seekable one can not cache so maybe do not do it. -- nosy: +grubert Added file: http://bugs.python.org/file22472/multiple_dump_load_test.patch ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Terry J. Reedy : -- versions: -Python 3.4 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip : Added file: http://bugs.python.org/file22440/ab1c38ffb8d4.diff ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip : Removed file: http://bugs.python.org/file22314/marshal-patch2.diff ___ Python tracker ___ ___ Python-bugs-list mailin

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip : Removed file: http://bugs.python.org/file22290/marshal-patch.diff ___ Python tracker ___ ___ Python-bugs-list mailing

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip : Added file: http://bugs.python.org/file22411/9e367c8fd949.diff ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by R. David Murray : -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: > Benjamin Peterson added the comment: > > I think you're right about playing with the bare fd being too fragile. I > think >a simpler solution is to read say 1024 bytes at a time and buffer it >internally. Doesn't this suffer from a similar problem? Name

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson added the comment: I think you're right about playing with the bare fd being too fragile. I think a simpler solution is to read say 1024 bytes at a time and buffer it internally. -- ___ Python tracker

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: @Benjamin: I missed commenting on your "Why not 0?", but here's the reasoning: one can't assume that the file only contains one object to be read, at the beginning of the file. It may be that some data is being written to file using marshal.dump, interspersed wi

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: > Benjamin Peterson added the comment: > > assert(newpos != NULL) > > That's because the call is failing. Why? > It's seemingly because the Python code did a seek (in Python) which was not communicated to the FILE object; after reading all objects from the

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson added the comment: 2011/6/19 Vinay Sajip : > > Vinay Sajip added the comment: > > This seems a bit hacky, and I'm not sure how reliable it is. I added this > after the read_object call: > >    if (is_file) { >        PyObject * newpos; >        int cp, np; > >        cp = fte

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: This seems a bit hacky, and I'm not sure how reliable it is. I added this after the read_object call: if (is_file) { PyObject * newpos; int cp, np; cp = ftell(rf.fp); newpos = PyObject_CallMethod(f, "seek", "ii", cp, SEEK_SET)

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson added the comment: 2011/6/19 Vinay Sajip : > > Vinay Sajip added the comment: > > The problem with calling fileno() and fdopen() is that you bypass the > buffering information held in BufferedIOReader. The first call works, but the > FILE * pointer is now positioned at 4K, r

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: The problem with calling fileno() and fdopen() is that you bypass the buffering information held in BufferedIOReader. The first call works, but the FILE * pointer is now positioned at 4K, rather than just past the end of the object just read. The next call fails

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson added the comment: 2011/6/18 Vinay Sajip : > > Vinay Sajip added the comment: > >> Benjamin Peterson added the  comment: >> >  Vinay Sajip added the  comment: >> > >> > Sorry I'm being dense, but which file object do you  mean? >> >> The python file  object. > > Do you mean

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: > Benjamin Peterson added the comment: > > Vinay Sajip added the comment: > > > > Sorry I'm being dense, but which file object do you mean? > > The python file object. Do you mean special-case handling of the circumstance when the file-like object being m

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson added the comment: 2011/6/18 Vinay Sajip : > > Vinay Sajip added the comment: > > Sorry I'm being dense, but which file object do you mean? The python file object. -- ___ Python tracker ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: Sorry I'm being dense, but which file object do you mean? -- ___ Python tracker ___ ___ Python-bugs-li

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson added the comment: Why can't you just call fileno() on the file object? -- ___ Python tracker ___ ___ Python-bugs-l

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Georg Brandl added the comment: Any reviewers? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: For some reason, "Create Patch" is failing with a [Errno 2] No such file or directory: '/home/roundup/trackers/tracker/cpython/Doc/Makefile' I've logged an issue on the meta tracker: http://psf.upfronthosting.co.za/roundup/meta/issue405 -- ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: Patch is now in my public sandbox on hg.python.org. -- hgrepos: +26 ___ Python tracker ___ ___ Python-

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Vinay Sajip : -- stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: Attached is an improved patch: 1. It's against the 3.2 branch, rather than default. 2. It has an added multiple load test in test_marshal.py. 3. There's more error checking for an EOF condition. 4. I've removed tab chars and used /* C89 comments */. Of course it'

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: Another thought about buffering, which might be a bit of a show-stopper: if in a particular load() call we read ahead for buffering purposes, we've altered the underlying stream (which might not be seekable to allow position restoring). The next call to load() w

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: @Amaury: Re formatting and comment style - I will address this. The diff I posted is to the default branch rather than 3.2, though it won't change the main substance of the patch. Re. performance and buffering: I agree this could have an impact (although string

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Georg Brandl added the comment: Sounds blocker-ish enough to me. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscr

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Changes by Barry A. Warsaw : -- nosy: +georg.brandl priority: normal -> release blocker ___ Python tracker ___ ___ Python-bugs-list ma

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Benjamin Peterson added the comment: It should probably be buffered at 512 bytes or some other reasonable number. -- nosy: +benjamin.peterson ___ Python tracker ___

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Amaury Forgeot d'Arc added the comment: - Please replace tabs characters by space - "//" comments are not accepted by some picky C89 compilers Also, calling f.read(1) for each character may have a large performance impact. I don't know how this can be solved though. -- _

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Vinay Sajip added the comment: I've added a patch which I think fixes marshal.load(), and would be grateful for a review - it's my first C patch for Python. The test_marshal test passes, though I haven't yet added a test for the failing scenario. -- keywords: +patch Added file: http:/

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

Amaury Forgeot d'Arc added the comment: Sadly, marshal.load() looks broken: - The function starts with the comment /* XXX Quick hack -- need to do this differently */ - It starts by calling f.read() which consumes the whole file (and explains the issue reported here) - The code was proba

[issue12291] file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3

New submission from Vinay Sajip : The attached file 'data.bin' was written using Python 3.2. It can be read by Python 2.7, but in 3.2 and 3.3, after the first object is read, the file pointer is positioned at EOF, causing an error on subsequent reads. A simple test script 'marshtest.py' is bel