[issue15546] Iteration breaks with bz2.open(filename,'rt')

Roundup Robot added the comment: New changeset 0f25119ceee8 by Serhiy Storchaka in branch '3.2': #15546: Fix GzipFile.peek()'s handling of pathological input data. http://hg.python.org/cpython/rev/0f25119ceee8 -- ___ Python tracker

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Nadeem Vawda added the comment: No, if _read() is called once the file is already at EOF, it raises an EOFError (http://hg.python.org/cpython/file/8c07ff7f882f/Lib/gzip.py#l433), which will then break out of the loop. -- ___ Python tracker

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Serhiy Storchaka added the comment: I have a doubts. Is it not a dead cycle if the end of the compressed data will happen on the end of reading block? Maybe instead of "while self.extrasize <= 0:" worth to write "while self.extrasize <= 0 and self.fileobj is not None:"? -- ___

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Roundup Robot added the comment: New changeset 8c07ff7f882f by Nadeem Vawda in branch 'default': #15546: Also fix GzipFile.peek(). http://hg.python.org/cpython/rev/8c07ff7f882f -- ___ Python tracker ___

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Nadeem Vawda added the comment: Before these fixes, it looks like all three classes' peek() methods were susceptible to the same problem as read1(). The fixes for BZ2File.read1() and LZMAFile.read1() should have fixed peek() as well; both methods are implemented in terms of _fill_buffer(). Fo

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Serhiy Storchaka added the comment: What about peek()? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Nadeem Vawda added the comment: Done. Thanks for the bug report, David. -- resolution: -> fixed stage: -> committed/rejected status: open -> closed ___ Python tracker ___

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Roundup Robot added the comment: New changeset 5284e65e865b by Nadeem Vawda in branch 'default': #15546: Fix {GzipFile,LZMAFile}.read1()'s handling of pathological input data. http://hg.python.org/cpython/rev/5284e65e865b -- ___ Python tracker

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Nadeem Vawda added the comment: OK, BZ2File should now be fixed. It looks like LZMAFile and GzipFile may be susceptible to the same problem; I'll push fixes for them shortly. -- ___ Python tracker _

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Roundup Robot added the comment: New changeset cdf27a213bd2 by Nadeem Vawda in branch 'default': #15546: Fix BZ2File.read1()'s handling of pathological input data. http://hg.python.org/cpython/rev/cdf27a213bd2 -- nosy: +python-dev ___ Python tracker

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Serhiy Storchaka added the comment: I encountered this when implemented bzip2 support in zipfile (issue14371). I solved this also by rewriting read and read1 to make as many reads from the underlying file as necessary to return a non-empty result. -- nosy: +storchaka _

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Antoine Pitrou added the comment: > I propose amending BZ2File.read1() to make as many reads > from the underlying file as necessary to return a non-empty result. Agreed. IMO, read1()'s contract should be read as a best-effort thing, not an absolute guarantee. Returning an empty string when the

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Nadeem Vawda added the comment: The cause of this problem is that BZ2File.read1() sometimes returns b"", even though the file is not at EOF. This happens when the underlying BZ2Decompressor cannot produce any decompressed data from just the block passed to it in _fill_buffer(); in this case, i

[issue15546] Iteration breaks with bz2.open(filename,'rt')

David Beazley added the comment: File attached.The file can be read in its entirety in binary mode. -- Added file: http://bugs.python.org/file26673/access-log-0108.bz2 ___ Python tracker ___

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Nadeem Vawda added the comment: I can't seem to reproduce this with an up-to-date checkout from Mercurial: >>> import bz2 >>> g = bz2.open('access-log-0108.bz2','rt') >>> next(g) '140.180.132.213 - - [24/Feb/2008:00:08:59 -0600] "GET /ply/ply.html HTTP/1.1" 200 97238\n' (where

[issue15546] Iteration breaks with bz2.open(filename,'rt')

Changes by Antoine Pitrou : -- nosy: +nadeem.vawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue15546] Iteration breaks with bz2.open(filename,'rt')

New submission from David Beazley: The bz2 library in Python3.3b1 doesn't support iteration for text-mode properly. Example: >>> f = bz2.open('access-log-0108.bz2') >>> next(f) # Works b'140.180.132.213 - - [24/Feb/2008:00:08:59 -0600] "GET /ply/ply.html HTTP/1.1" 200 97238\n' >>> g =