[issue1519] async_chat.__init__() parameters
New submission from Nadeem Vawda: The __init__() function for asynchat.async_chat doesn't allow the caller to specify a channel map. I thought it would make sense to add an optional 'map' parameter, for consistency with asyncore.dispatcher. If the parameter is not specified, asyncore.dispatcher.__init__() will default to using the global map, which is the current behaviour. -- components: Library (Lib) files: asynchat.patch messages: 57930 nosy: nvawda severity: minor status: open title: async_chat.__init__() parameters type: behavior versions: Python 2.5 Added file: http://bugs.python.org/file8822/asynchat.patch __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1519> __ asynchat.patch Description: Binary data ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Added file: http://bugs.python.org/file23129/fe2c9998f329.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: I've attached another patch (fe2c9998f329.diff) with a more complete implementation of the lzma module. All that's left now is to write the documentation, and make sure that the module can build on Windows. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12306] zlib: Expose zlibVersion to query runtime version of zlib
Nadeem Vawda added the comment: I've committed your patches. I took the liberty of removing the "versionadded" tag for ZLIB_VERSION; I don't think many people will need to worry about compatibility with Python 1.5 ;-) Once again, thanks for the patches! -- assignee: -> nadeem.vawda resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue12306> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12678] test_packaging and test_distutils failures under Windows
Nadeem Vawda added the comment: Issue 12785 has a patch for the test_database failures. -- ___ Python tracker <http://bugs.python.org/issue12678> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Added file: http://bugs.python.org/file23158/95016f363e6c.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Added file: http://bugs.python.org/file23159/3017ee1d548d.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Removed file: http://bugs.python.org/file23158/95016f363e6c.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Removed file: http://bugs.python.org/file23063/f3cf187208ea.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Removed file: http://bugs.python.org/file23129/fe2c9998f329.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: I've attached another patch that includes proper ReST documentation, a few more tests, and some fixups following Ezio and Amaury's reviews. -- stage: needs patch -> patch review ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Added file: http://bugs.python.org/file23168/591277fe6b4a.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Removed file: http://bugs.python.org/file23159/3017ee1d548d.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Thanks for getting the ball rolling with that. I'll want to try and familiarize myself with the code for tarfile module before I start changing stuff, but those points shouldn't be hard to fix, one way or the other. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Ah, sorry about that. It probably wasn't necessary to regenerate the patch after I addressed the comments in each review. The patch should be stable now; I don't plan to make any further changes for a while. (At some point I'll want to add a few more tests, and get the module to compile on Windows. But that can wait.) -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Nadeem Vawda added the comment: Ozan: Thanks for taking the time to backport the patch. Unfortunately, as Éric said, 2.7 is in maintenance mode, so it no longer receives new features. -- ___ Python tracker <http://bugs.python.org/issue1625> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Nadeem Vawda added the comment: > This is all fine and well, but this is clearly a bug and not a feature. No, it is not at all clear that this is a bug. I agree that this is a desirable capability to have, but nowhere does the module claim to support multi-stream files. Nor is it an inherent feature of the underlying bzip2 library that we are failing to expose to users. > [...] python 2.x users will never be able to extract multiple-stream bz2 > files. Incorrect. It is perfectly possible to extract a multi-stream bz2 file in 2.x - you just need to open it with open() and decompress the data using BZ2Decompressor (pretty much the same way that 3.3's BZ2File does it). If there is really a large demand for these facilities in 2.x, I would be willing to port 3.3's BZ2File implementation to 2.x and make it available on PyPI. But this patch is not going in to the 2.7 stdlib; it is simply not the sort of behavior change that is acceptable in a bugfix release. -- ___ Python tracker <http://bugs.python.org/issue1625> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12804] "make test" fails on systems without internet access
Nadeem Vawda added the comment: > Change reverted. "make test" should run a comprehensive test of > Python's facilities Fair enough. > If it is easy to detect network availability programmatically, we could > just use the skip system. +1. I don't know if there is a reasonable way to do this, but if so, that would be the best solution. -- stage: -> needs patch title: make test should not enable the urlfetch resource -> "make test" fails on systems without internet access ___ Python tracker <http://bugs.python.org/issue12804> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Thanks for investigating the Windows situation. > - liblzma can't be compiled by Visual Studio: too many C99 isms, mostly > variables declared in the middle of a block. It's doable for sure, but it's a > lot of work. I don't think that creating our own MSVC-friendly fork of liblzma is really an option. Over and above the work of porting it in the first place (and all the opportunities for bugs to creep in along the way), we'd also have to worry about keeping up to date with upstream changes. I believe we currently do something similar with libffi (for ctypes), and the impression I've gotten is that it's caused a lot of trouble. > - The way recommended by XZ is to use a precompiled liblzma.dll; Then it > should be easy to build an extension module, but its would be the first time > that we distribute an extension module which needs a non-system DLL. Is it > enough to copy it next to _lzma.pyd? Is there some work to do in the > installer? I would guess that this is sufficient, but my knowledge of how Windows DLLs work is minimal. Could someone with more platform knowledge weigh in on whether this would work (and if there are any problems it might cause)? -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12804] "make test" fails on systems without internet access
Nadeem Vawda added the comment: Oh, neat. I'll take a look at that when I get a chance. -- assignee: -> nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12804> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Wow, this discussion has gotten quite busy while I've been travelling... Martin, could you explain what the problems are with bundling a precompiled DLL for Windows? I am willing to do the work of getting liblzma to compile with VS if necessary, but I don't know how receptive the upstream maintainer will be to the changes. If I can explain how lack of VS support is a problem for us, the request should carry more weight. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Hmm... according to http://git.tukaani.org/?p=xz.git;a=blob;f=windows/README-Windows.txt;hb=HEAD#l80, the MinGW-compiled static libs *can* be used with MSVC. Not sure how reliable the information is, but it's worth a try at least. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13150] Most of Python's startup time is sysconfig
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue13150> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Awesome stuff! I'll post an updated patch during the course of the day. Martin: I've been having problems with Rietveld lately, so I'm posting my replies to your comments here instead. >> Modules/_lzmamodule.c:115: return _PyBytes_Resize(buf, size + BIGCHUNK); > This has quadratic performance. Correct. I copied the algorithm from _io.FileIO, under the assumption that there was a reason for not using a simpler O(n log n) doubling strategy. Do you know of any reason for this? Or is it safe to ignore it? >> Modules/_lzmamodule.c:364: Py_BEGIN_ALLOW_THREADS > It seems that the Windows version at least is not thread-safe. If so, you > would need an LZMA lock when releasing the GIL. Does the class need to be thread-safe, though? ISTM that there isn't any sensible use case for having two threads feeding data through the same compressor concurrently. (If we *do* want thread-safety, then it doesn't matter whether the underlying lib is internally thread-safe or not. We would still need to guard against the possibility of the _lzmamodule.c code in one thread modifying the lzma_stream's input or output pointer while lzma_code is operating on the stream's data in another thread.) -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13159] _io.FileIO uses a quadratic-time buffer growth algorithm
New submission from Nadeem Vawda :
As mentioned in issue 6715, the buffer growth strategy used by _io.FileIO
has quadratic running time if each reallocation is O(n). The code in
question is new_buffersize() from Modules/_io/fileio.c:
if (currentsize > SMALLCHUNK) {
/* Keep doubling until we reach BIGCHUNK;
then keep adding BIGCHUNK. */
if (currentsize <= BIGCHUNK)
return currentsize + currentsize;
else
return currentsize + BIGCHUNK;
}
return currentsize + SMALLCHUNK;
Is there a reason for this? One possible improvement would be to instead
use the same formula as list.resize() in Objects/listobject.c:
new_allocated = (newsize >> 3) + (newsize < 9 ? 3 : 6);
which has amortized O(n) running time behaviour.
Your thoughts?
--
components: IO
messages: 145403
nosy: benjamin.peterson, nadeem.vawda, pitrou, stutzbach
priority: normal
severity: normal
stage: needs patch
status: open
title: _io.FileIO uses a quadratic-time buffer growth algorithm
type: performance
versions: Python 3.3
___
Python tracker
<http://bugs.python.org/issue13159>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: > To elaborate on the policy: giving bogus data in cases of race > conditions is ok; crashing the interpreter or corrupting memory > is not. If bogus data is given, it would be useful if the bogosity > can be specified (e.g. when multiple threads read from the same > POSIX file concurrently, they also get bogus data, but in a manner > where each input byte is given to exactly one thread). OK, that makes sense. My next patch will include per-compressor locks for LZMACompressor and LZMADecompressor. > To elaborate: ISTM that it's actually a bug in FileIO. I've filed issue 13159 proposing that FileIO be fixed. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13159] _io.FileIO uses a quadratic-time buffer growth algorithm
Nadeem Vawda added the comment: I've attached a patch that makes the suggested change to FileIO, and also to _bz2.BZ2Compressor/Decompressor (which currently have the same issue). -- keywords: +patch Added file: http://bugs.python.org/file23389/buffer-growth.diff ___ Python tracker <http://bugs.python.org/issue13159> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13159] _io.FileIO uses a quadratic-time buffer growth algorithm
Changes by Nadeem Vawda : -- stage: needs patch -> patch review ___ Python tracker <http://bugs.python.org/issue13159> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Added file: http://bugs.python.org/file23393/bdf0afbbbd80.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Removed file: http://bugs.python.org/file23168/591277fe6b4a.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Patch update time! Noteworthy changes: * Windows build support, thanks to Amaury * Thread safety for LZMACompressor and LZMADecompressor, by means of per-instance locks * O(n) buffer growth strategy I've tried running the tests on Windows, but I've been getting failures that don't look like they're related to the new code. I think they are due to issue 7443. I'll try and tweak my Windows setup a bit over the weekend to see if I can get things working smoothly. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13159] _io.FileIO uses a quadratic-time buffer growth algorithm
Nadeem Vawda added the comment: No problem :) -- assignee: -> nadeem.vawda resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue13159> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13194] zlib (de)compressobj copy() method missing on Windows
Nadeem Vawda added the comment: Confirmed for 3.3. I should be able to post a patch tomorrow. -- assignee: -> nadeem.vawda ___ Python tracker <http://bugs.python.org/issue13194> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: After disabling a bunch of services and skipping the tests that are failing on the Windows buildbots, it seems that the failures I was getting last week have disappeared. It looks like they were due to the Windows search indexer preventing temp directories from being deleted... >_> > I have imported xz-5.0.3 into the externals repository now. I tried to test this out, but it looks like liblzma.a is missing from the SVN repo. There should be files with that name in the bin_i486 and bin_x86-64 directories. Apart from this, is there anything else that needs to be done before this change can be committed to the main repo? -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13194] zlib (de)compressobj copy() method missing on Windows
Nadeem Vawda added the comment: Fixed for 2.7, 3.2 and 3.3. Thanks for the bug report! -- resolution: -> fixed stage: -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue13194> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13218] test_ssl failures on Ubuntu 11.10
New submission from Nadeem Vawda :
Since upgrading to Ubuntu 11.10, I've been getting the following failures in
test_ssl:
==
FAIL: test_protocol_sslv3 (test.test_ssl.ThreadedTests)
Connecting to an SSLv3 server with various client options
--
Traceback (most recent call last):
File "/home/nadeem/code/src/cpython/def/Lib/test/test_ssl.py", line 1415,
in test_protocol_sslv3
try_protocol_combo(ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv23, False)
File "/home/nadeem/code/src/cpython/def/Lib/test/test_ssl.py", line 1232,
in try_protocol_combo
ssl.get_protocol_name(server_protocol)))
AssertionError: Client protocol SSLv23 succeeded with server protocol SSLv3!
==
FAIL: test_protocol_tlsv1 (test.test_ssl.ThreadedTests)
Connecting to a TLSv1 server with various client options
--
Traceback (most recent call last):
File "/home/nadeem/code/src/cpython/def/Lib/test/test_ssl.py", line 1433,
in test_protocol_tlsv1
try_protocol_combo(ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv23, False)
File "/home/nadeem/code/src/cpython/def/Lib/test/test_ssl.py", line 1232,
in try_protocol_combo
ssl.get_protocol_name(server_protocol)))
AssertionError: Client protocol SSLv23 succeeded with server protocol TLSv1!
--
and this (presumably related) failure in test_nntplib:
==
ERROR: setUpClass (test.test_nntplib.NetworkedNNTP_SSLTests)
--
Traceback (most recent call last):
File "/home/nadeem/code/src/cpython/def/Lib/test/test_nntplib.py", line
291, in setUpClass
cls.server = cls.NNTP_CLASS(cls.NNTP_HOST, timeout=TIMEOUT,
usenetrc=False)
File "/home/nadeem/code/src/cpython/def/Lib/nntplib.py", line 1052, in
__init__
readermode=readermode, timeout=timeout)
File "/home/nadeem/code/src/cpython/def/Lib/nntplib.py", line 344, in
__init__
self.getcapabilities()
File "/home/nadeem/code/src/cpython/def/Lib/nntplib.py", line 380, in
getcapabilities
resp, caps = self.capabilities()
File "/home/nadeem/code/src/cpython/def/Lib/nntplib.py", line 546, in
capabilities
resp, lines = self._longcmdstring("CAPABILITIES")
File "/home/nadeem/code/src/cpython/def/Lib/nntplib.py", line 513, in
_longcmdstring
resp, list = self._getlongresp(file)
File "/home/nadeem/code/src/cpython/def/Lib/nntplib.py", line 464, in
_getlongresp
resp = self._getresp()
File "/home/nadeem/code/src/cpython/def/Lib/nntplib.py", line 437, in
_getresp
resp = self._getline()
File "/home/nadeem/code/src/cpython/def/Lib/nntplib.py", line 425, in
_getline
if not line: raise EOFError
EOFError
3.2 gives a different set of failures in test_ssl (with the same failure in
test_nntplib):
==
ERROR: test_constructor (test.test_ssl.ContextTests)
--
Traceback (most recent call last):
File "/home/nadeem/code/src/cpython/3.2/Lib/test/test_ssl.py", line 82,
in f
return func(*args, **kwargs)
File "/home/nadeem/code/src/cpython/3.2/Lib/test/test_ssl.py", line 345,
in test_constructor
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv2)
File "/home/nadeem/code/src/cpython/3.2/Lib/ssl.py", line 168, in __new__
return _SSLContext.__new__(cls, protocol)
ssl.SSLError: failed to allocate SSL context
==
ERROR: test_protocol (test.test_ssl.ContextTests)
--
Traceback (most recent call last):
File "/home/nadeem/code/src/cpython/3.2/Lib/test/test_ssl.py", line 82,
in f
return func(*args, **kwargs)
File "/home/nadeem/code/src/cpython/3.2/Lib/test/test_ssl.py", line 356,
in test_protocol
ctx = ssl.SSLContext(proto)
File "/home/nadeem/code/src/cpython/3.2/Lib/ssl.py", line 168, in __new__
return _SSLContext.__new__(cls, protocol)
ssl.SSLError: failed to allocate SSL context
==
ERROR: test_session_stats (test.test_ssl.Contex
[issue13218] test_ssl failures on Ubuntu 11.10
Nadeem Vawda added the comment: > Also, how about the system Python? Does it show the same test failures? Unfortunately, it seems that the system Python package doesn't install the test suite; if you try to run "python -m test.regrtest", it complains that it can't find the test modules. > Actually, I think you may just have to fix skip_if_broken_ubuntu_ssl() in > test_ssl.py. Could you give it a try? I'll take a look over the weekend. -- ___ Python tracker <http://bugs.python.org/issue13218> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13218] test_ssl failures on Ubuntu 11.10
Nadeem Vawda added the comment: I get (1, 0, 0, 5, 15) for ssl.OPENSSL_VERSION_INFO. A trivial modification of Victor's patch doesn't seem to fix the failures for me, though :/ -- ___ Python tracker <http://bugs.python.org/issue13218> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13218] test_ssl failures on Ubuntu 11.10
Nadeem Vawda added the comment:
Sorry, that wasn't very coherent. What I meant to say is that I tried this:
if ((ssl.OPENSSL_VERSION_INFO, platform.linux_distribution()) in
[((0, 9, 8, 15, 15), ('debian', 'squeeze/sid', '')),
((1, 0, 0, 5, 15), ('debian', 'wheezy/sid', ''))]):
raise unittest.SkipTest("Patched Ubuntu OpenSSL breaks behaviour")
on the default branch, and I'm still getting the same failures.
--
___
Python tracker
<http://bugs.python.org/issue13218>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13218] test_ssl failures on Ubuntu 11.10
Nadeem Vawda added the comment: > Hmm... I guess Ubuntu may have aliased SSLv23 to SSLv3. Can someone take > a look at Ubuntu patches to OpenSSL? I don't know enough about OpenSSL to make sense of the actual headers on my system without spending more time on it, but it seems like Debian have disabled SSLv2 altogether <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=589706>. Could this be the source of the problem? -- ___ Python tracker <http://bugs.python.org/issue13218> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures under Windows
Nadeem Vawda added the comment:
> The test creates a temporary directory which is inserted at the head of
> sys.path.
> packaging.database.get_distribution should thus find Spamlib-0.1.dist-info.
> Can someone with a Windows install help me with this? Printing sys.path and
> os.listdir(sys.path[0]) would be a good start.
test_resources
(packaging.tests.test_command_install_data.InstallDataTestCase) ...
sys.path: ['c:\\users\\nadeem\\appdata\\local\\temp\\tmpi7pf1f\\tmp4gdwnp',
'',
'C:\\Users\\Nadeem\\Code\\python3\\python\\PCbuild\\python33_d.zip',
'C:\\Users\\Nadeem\\Code\\python3\\python\\DLLs',
'C:\\Users\\Nadeem\\Code\\python3\\python\\lib',
'C:\\Users\\Nadeem\\Code\\python3\\python\\PCbuild',
'C:\\Users\\Nadeem\\AppData\\Roaming\\Python\\Python33\\site-packages',
'C:\\Users\\Nadeem\\Code\\python3\\python',
'C:\\Users\\Nadeem\\Code\\python3\\python\\lib\\site-packages']
os.listdir(sys.path[0]): ['Spamlib-0.1.dist-info']
ERROR
I noticed that the test succeeds if you run only the InstallDataTestCase (with
"... -v -m InstallDataTestCase test_packaging"), but the sys.path info for this
case is the same as before (modulo the name of the temp directory):
test_resources
(packaging.tests.test_command_install_data.InstallDataTestCase) ...
sys.path: ['c:\\users\\nadeem\\appdata\\local\\temp\\tmpddi17y\\tmp1mnv9z',
'',
'C:\\Users\\Nadeem\\Code\\python3\\python\\PCbuild\\python33_d.zip',
'C:\\Users\\Nadeem\\Code\\python3\\python\\DLLs',
'C:\\Users\\Nadeem\\Code\\python3\\python\\lib',
'C:\\Users\\Nadeem\\Code\\python3\\python\\PCbuild',
'C:\\Users\\Nadeem\\AppData\\Roaming\\Python\\Python33\\site-packages',
'C:\\Users\\Nadeem\\Code\\python3\\python',
'C:\\Users\\Nadeem\\Code\\python3\\python\\lib\\site-packages']
os.listdir(sys.path[0]): ['Spamlib-0.1.dist-info']
ok
Additionally, I've been getting a similar failure on my Ubuntu machine:
==
ERROR: test_resources
(packaging.tests.test_command_install_data.InstallDataTestCase)
--
Traceback (most recent call last):
File
"/home/nadeem/code/src/cpython/def/Lib/packaging/tests/test_command_install_data.py",
line 129, in test_resources
with packaging.database.get_file('Spamlib', 'spamd') as fp:
File "/home/nadeem/code/src/cpython/def/Lib/packaging/database.py", line
649, in get_file
return open(get_file_path(distribution_name, relative_path),
File "/home/nadeem/code/src/cpython/def/Lib/packaging/database.py", line
644, in get_file_path
raise LookupError('no distribution named %r found' % distribution_name)
LookupError: no distribution named 'Spamlib' found
Output from Ubuntu machine:
test_resources
(packaging.tests.test_command_install_data.InstallDataTestCase) ...
sys.path: ['/tmp/tmpubkdye/tmpdmdybx',
'',
'/usr/local/lib/python33.zip',
'/home/nadeem/code/src/cpython/def/Lib',
'/home/nadeem/code/src/cpython/def/Lib/plat-linux',
'/home/nadeem/code/src/cpython/def/build/lib.linux-x86_64-3.3',
'/home/nadeem/.local/lib/python3.3/site-packages',
'/usr/local/lib/python3.3/site-packages']
os.listdir(sys.path[0]): ['Spamlib-0.1.dist-info']
ERROR
--
nosy: +nadeem.vawda
___
Python tracker
<http://bugs.python.org/issue13193>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12367] select.error has no errno attribute
Nadeem Vawda added the comment: The AMD64 FreeBSD 8.2 3.x buildbot has been experiencing sporadic lockups since changeset 8bbfb24d4824 was committed: http://www.python.org/dev/buildbot/all/builders/AMD64%20FreeBSD%208.2%203.x/builds/1174/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/AMD64%20FreeBSD%208.2%203.x/builds/1180/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/AMD64%20FreeBSD%208.2%203.x/builds/1185/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/AMD64%20FreeBSD%208.2%203.x/builds/1190/steps/test/logs/stdio The tracebacks say that the lockups are happening in test_errno, which is weird because the test is supposed to be skipped on FreeBSD. Is it possible that sys.platform is returning something that doesn't start with "freebsd" on this buildbot? -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12367> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12367] select.error has no errno attribute
Nadeem Vawda added the comment: Sorry, please disregard my last message - I confused the commit times for 8bbfb24d4824 and f6b8e4226260. There haven't been any further lockups since f6b8e4226260 was committed. -- ___ Python tracker <http://bugs.python.org/issue12367> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Removed file: http://bugs.python.org/file23393/bdf0afbbbd80.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Added file: http://bugs.python.org/file23466/371a133b770a.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13230] test_resources fails
Nadeem Vawda added the comment: This seems to be a duplicate of issue 13193. -- nosy: +nadeem.vawda resolution: -> duplicate stage: needs patch -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue13230> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures under Windows
Nadeem Vawda added the comment: Issue 13230 (closed as duplicate) reports another Linux failure in test_resources. Changing the title, since it isn't Windows-specific. -- nosy: +anikom15 stage: -> needs patch ___ Python tracker <http://bugs.python.org/issue13193> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Changes by Nadeem Vawda : -- title: test_packaging and test_distutils failures under Windows -> test_packaging and test_distutils failures ___ Python tracker <http://bugs.python.org/issue13193> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13218] test_ssl failures on Ubuntu 11.10
Nadeem Vawda added the comment: > none of the buildbots are having this problem. Actually, it turns out the Ubuntu ARM builder is hitting the same failures. First failure for each branch was on 14 October (the day after 11.10 came out): http://www.python.org/dev/buildbot/all/builders/ARM%20Ubuntu%203.x/builds/16 http://www.python.org/dev/buildbot/all/builders/ARM%20Ubuntu%203.2/builds/9 http://www.python.org/dev/buildbot/all/builders/ARM%20Ubuntu%202.7/builds/8 -- ___ Python tracker <http://bugs.python.org/issue13218> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Nadeem Vawda added the comment:
> Spamlib-0.1.dist-info is in a directory on sys.path, so the error makes no
> sense to me. Can you inspect the content of that directory? (i.e os.listdir
> and file contents) Also, if you can interrupt the test to get a Python or
> pdb shell, could you try this:
>
> >>> from packaging.database import Distribution
> >>> d = Distribution('path/to/Spamlib-0.1.dist-info')
> >>> d.metadata
> >>> d.list_distinfo_files()
> >>> d.list_installed_files()
(Pdb) p install_dir
'c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b'
(Pdb) from packaging.database import Distribution
(Pdb) _d = Distribution(os.path.join(install_dir, "Spamlib-0.1.dist-info"))
(Pdb) p _d
(Pdb) p _d.metadata
(Pdb) p dict(_d.metadata.items())
{'Name': 'Spamlib', 'License': 'UNKNOWN', 'Author': 'UNKNOWN',
'Metadata-Version': '1.0',
'Home-page': 'UNKNOWN', 'Summary': 'UNKNOWN', 'Platform': [], 'Version': '0.1',
'Keywords': [''], 'Author-email': 'UNKNOWN', 'Description': 'UNKNOWN'}
(Pdb) p list(_d.list_distinfo_files())
['c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\METADATA',
'c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\INSTALLER',
'c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\REQUESTED',
'c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\RESOURCES',
'c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\RECORD']
(Pdb) p list(_d.list_installed_files())
[('c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmpo8yw3_\\spamd',
'd13e6156ce78919a981e424b2fdcd974', '15'),
('c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\METADATA',
'8c9f474663f774563ded42983eb22447', '182'),
('c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\INSTALLER',
'44e3fde05f3f537ed85831969acf396d', '9'),
('c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\REQUESTED',
'd41d8cd98f00b204e9800998ecf8427e', '0'),
('c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\RESOURCES',
'1c5aec043865966e17b1044e20087b9d', '68'),
('c:\\users\\nadeem\\appdata\\local\\temp\\tmpe96yhv\\tmp9ga04b\\Spamlib-0.1.dist-info\\RECORD',
'', '')]
File contents:
spamd [no EOL]:
# Python script
METADATA:
Metadata-Version: 1.0
Name: Spamlib
Version: 0.1
Summary: UNKNOWN
Description: UNKNOWN
Keywords:
Home-page: UNKNOWN
Author: UNKNOWN
Author-email: UNKNOWN
License: UNKNOWN
INSTALLER [no EOL]:
distutils
REQUESTED is empty
RESOURCES:
spamd,c:\users\nadeem\appdata\local\temp\tmpe96yhv\tmpo8yw3_\spamd
RECORD:
c:\users\nadeem\appdata\local\temp\tmpe96yhv\tmpo8yw3_\spamd,d13e6156ce78919a981e424b2fdcd974,15
c:\users\nadeem\appdata\local\temp\tmpe96yhv\tmp9ga04b\Spamlib-0.1.dist-info\METADATA,8c9f474663f774563ded42983eb22447,182
c:\users\nadeem\appdata\local\temp\tmpe96yhv\tmp9ga04b\Spamlib-0.1.dist-info\INSTALLER,44e3fde05f3f537ed85831969acf396d,9
c:\users\nadeem\appdata\local\temp\tmpe96yhv\tmp9ga04b\Spamlib-0.1.dist-info\REQUESTED,d41d8cd98f00b204e9800998ecf8427e,0
c:\users\nadeem\appdata\local\temp\tmpe96yhv\tmp9ga04b\Spamlib-0.1.dist-info\RESOURCES,1c5aec043865966e17b1044e20087b9d,68
c:\users\nadeem\appdata\local\temp\tmpe96yhv\tmp9ga04b\Spamlib-0.1.dist-info\RECORD,,
> Where should be the distro?
You won't find it in the hg repository - it gets created by the test, and
then deleted afterwards.
> The changeset was:
> $ hg tip
> changeset: 73075:d4839fea4a5a
> [...]
That's not possible - these failures have been around for over a week now,
and that changeset was only committed yesterday.
>From running the tests locally, it looks like test_resources was failing on
Windows since it was first introduced in 1f3459b08298. This didn't show up so
clearly on the buildbots because the build was broken at the time.
--
___
Python tracker
<http://bugs.python.org/issue13193>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Nadeem Vawda added the comment: I've been able to consistently reproduce the test_resources failure on Ubuntu 11.10 64-bit, FWIW. -- ___ Python tracker <http://bugs.python.org/issue13193> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13218] test_ssl failures on Debian/Ubuntu
Nadeem Vawda added the comment: > New changeset 3c225f938dae by Barry Warsaw in branch '2.7': > - Issue #13218: Fix test_ssl failures on Debian/Ubuntu. > http://hg.python.org/cpython/rev/3c225f938dae This changeset appears to have broken a number of the 2.7 Linux buildbots: http://www.python.org/dev/buildbot/all/builders/AMD64%20Gentoo%20Wide%202.7/builds/861 http://www.python.org/dev/buildbot/all/builders/x86%20Gentoo%202.7/builds/287 http://www.python.org/dev/buildbot/all/builders/x86%20Gentoo%20Non-Debug%202.7/builds/245 http://www.python.org/dev/buildbot/all/builders/x86%20Ubuntu%20Shared%202.7/builds/1147 http://www.python.org/dev/buildbot/all/builders/ARM%20Ubuntu%202.7/builds/24 All of the errors look something like: == ERROR: test_protocol_sslv3 (test.test_ssl.ThreadedTests) Connecting to an SSLv3 server with various client options -- Traceback (most recent call last): File "/home/buildbot/buildarea/2.7.ochtman-gentoo-amd64/build/Lib/test/test_ssl.py", line 75, in f return func(*args, **kwargs) File "/home/buildbot/buildarea/2.7.ochtman-gentoo-amd64/build/Lib/test/test_ssl.py", line 1029, in test_protocol_sslv3 client_options=ssl.OP_NO_SSLv3) AttributeError: 'module' object has no attribute 'OP_NO_SSLv3' == ERROR: test_protocol_tlsv1 (test.test_ssl.ThreadedTests) Connecting to a TLSv1 server with various client options -- Traceback (most recent call last): File "/home/buildbot/buildarea/2.7.ochtman-gentoo-amd64/build/Lib/test/test_ssl.py", line 75, in f return func(*args, **kwargs) File "/home/buildbot/buildarea/2.7.ochtman-gentoo-amd64/build/Lib/test/test_ssl.py", line 1044, in test_protocol_tlsv1 client_options=ssl.OP_NO_TLSv1) AttributeError: 'module' object has no attribute 'OP_NO_TLSv1' The ARM Ubuntu builder gets two additional errors, saying that ssl.PROTOCOL_SSLv2 doesn't exist either. -- ___ Python tracker <http://bugs.python.org/issue13218> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6434] buffer overflow in Zipfile when wrinting more than 2gig file
Nadeem Vawda added the comment: Marking as duplicate. -- resolution: -> duplicate stage: needs patch -> committed/rejected status: open -> closed superseder: -> zipfile writes incorrect local file header for large files in zip64 type: crash -> behavior ___ Python tracker <http://bugs.python.org/issue6434> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9720] zipfile writes incorrect local file header for large files in zip64
Nadeem Vawda added the comment: Issue 6434 was marked as a duplicate of this issue. -- nosy: +Paul, amaury.forgeotdarc, enlavin, lambacck, nadeem.vawda, segfault42 stage: -> needs patch versions: +Python 3.3 -Python 3.1 ___ Python tracker <http://bugs.python.org/issue9720> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Nadeem Vawda added the comment: > I added the packaging.database.clear_cache() at the end of setUp(); that > works, too. That fixes the InstallDataTestCase failure for me as well (on both Ubuntu 11.10 64-bit and Windows 7 64-bit). > I get the opposite failure to Nadeem as far as > InstallDataTestCase.test_resources: it works on Ubuntu 64-bit, but fails on > 32-bit. FTR, I haven't checked this on Ubuntu 32-bit. It's rather odd for the failure to be architecture-dependent. Are these two machines running different versions of Ubuntu, perhaps? -- ___ Python tracker <http://bugs.python.org/issue13193> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Nadeem Vawda added the comment: How strange. I've had no problem reproducing the failures; they've occurred every time I've run test_packaging since the bug was opened (on both Ubuntu and Windows). And the Windows buildbots seem to be failing consistently as well... -- ___ Python tracker <http://bugs.python.org/issue13193> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13309] test_time fails: time data 'LMT' does not match format '%Z'
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue13309> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13193] test_packaging and test_distutils failures
Nadeem Vawda added the comment: I'm no longer getting the failures on either Ubuntu or Windows (and the Windows buildbots are now green), so tentatively marking this as fixed. Feel free to reopen if something is still broken. -- resolution: -> fixed stage: needs patch -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue13193> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13396] new method random.getrandbytes()
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue13396> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Everything is ready to go, as far as I'm concerned. Once I've tested the Windows build using the binary from the externals repository, I'll be ready to commit it to the main repository. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: I've tested the Windows build, and it works fine. If there are no objections, I'll commit on Saturday. Btw, what is the preferred channel for communicating with buildbot owners? I'll want to send out an email asking that all the *nix bots have liblzma-dev installed, so that they can actually run the tests... -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Added file: http://bugs.python.org/file23732/9276fc685c05.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Changes by Nadeem Vawda : Removed file: http://bugs.python.org/file23466/371a133b770a.diff ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13453] Tests and network timeouts
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue13453> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Victor: Thanks for the review; I've replied to your comments and updated the patch. Let me know what you think of the changes. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: The patch has been committed. I'm leaving this issue open for a while longer, until more of the buildbots are building and testing the module. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: >> Not meaning to sound petty, but wouldn't it be common etiquette to >> retain some original copyright notice from original code intact..? > > It seemed to me that Nadeem had rewritten everything from scratch. Is > there any code of yours in the current module? That is correct. Based on my experience with the bz2 module, rewriting from scratch seemed to make more sense. >> Oh well, I thought that I'd still earned a note with some slight > credit at least > > I completely agree. Sometimes people get credit for simple bug fixes > (count me among them) so the author of the first working implementation > deserves some recognition IMO. Of course. How does this look? diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -502,6 +502,7 @@ Peter van Kampen Rafe Kaplan Jacob Kaplan-Moss +Per Øyvind Karlsen Lou Kates Hiroaki Kawai Sebastien Keim diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -400,6 +400,7 @@ --- - Issue #6715: Add a module 'lzma' for compression using the LZMA algorithm. + Thanks to Per Øyvind Karlsen for the initial implementation. - Issue #13487: Make inspect.getmodule robust against changes done to sys.modules while it is iterating over it. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: > Instead of duplicating the list of archiving/compression modules in > each doc, what about only linking to the shutil doc for archives and > the archiving.rst file? Sure, go ahead. I actually hadn't realized that each section of the library docs had a "sub-index" page like that until now... -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5411] add xz compression support to shutil
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue5411> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: > As the docs for zlib, gzip, bz2, lzma, zipfile and tarfile are in the > archiving subsection, there’s already a link to the subsection index, > so I just removed the “See also zlib, etc.” lines (except for the link > from zlib to gzip). I added a link from archiving.rst to shutil and > more links and reST targets in shutil. Looks good to me. > The entry in Misc/ACKS, Doc/whatsnew/3.3.rst and the commit message > should be enough. Lately I’ve noticed some attributions in NEWS, but > it’s usually not done (as redundant). According to Doc/whatsnew/3.3.rst: "The maintainer will go through Misc/NEWS periodically and add changes; it's therefore more important to add your changes to Misc/NEWS than to this file." > what about a mention in lzmamodule.c? Done and committed. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: > I’ll commit my doc patch to all branches later. OK, great. > I checked the 4 red 3.3 builbots and they can’t build _lzma. For the record, the 3.x buildbots that currently aren't able to build the _lzma module are: * x86 Ubuntu Shared * sparc solaris10 gcc * AMD64 Snow Leopard 2 * x86 debian parallel * ARM Ubuntu * x86 Tiger * AMD64 OpenIndiana * x86 OpenIndiana * x86 FreeBSD 7.2 * x86 FreeBSD 6.4 Additionally, the AMD64 debian bigmem, PPC Tiger and PPC Leopard builders have been offline/broken since before changeset 74d182cf0187 was committed. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: > Ok, I've e-mailed the owners. Thanks. I was just thinking I should send a reminder, in case they missed the note in my announcement on python-dev. > Will someone write an entry in the "what's new" file for 3.3? Done -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5689] Support xz compression in tarfile module
Nadeem Vawda added the comment: Great stuff! I'll close this issue along with issue 6715 once the buildbot stuff is all sorted out. -- ___ Python tracker <http://bugs.python.org/issue5689> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6715] xz compressor support
Nadeem Vawda added the comment: Yes, almost half of the buildbots still don't have the xz-utils headers installed, and thus are not building/testing the lzma module. I've kept the issue open as a reminder to myself to follow up on this. -- ___ Python tracker <http://bugs.python.org/issue6715> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5689] Support xz compression in tarfile module
Nadeem Vawda added the comment: This failure seems to crop up often, but not on every run: http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/3941/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/3940/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/3937/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/3929/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/3921/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/3916/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/3914/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/3906/steps/test/logs/stdio I've been able to reproduce the failure on my own XP machine; I'll investigate it over the weekend. -- ___ Python tracker <http://bugs.python.org/issue5689> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5689] Support xz compression in tarfile module
Nadeem Vawda added the comment: Not to worry - as I said in my previous message, I can reproduce the error on my own XP machine. I also noticed that running test_tarfile alone doesn't trigger the errors, which leads me to suspect that the failure is due to some interaction with another test getting run before test_tarfile. I'm currently trying to determine what this test is. I suspect that the problem is at least partially caused by the fact that tarfile uses a default compresslevel of 9 for .tar.xz archives (rather than the recommended value of 6). According to the man page for the xz tool <http://manpages.ubuntu.com/manpages/lucid/man1/xz.1.html>, using a compresslevel of 9 can result in memory usage of up to 800MB during compression, which is a significant fraction of the bot's 2GB of RAM. (I suppose it would be a good idea to mention this in the documentation for the lzma module, so users won't get bitten by this...) -- ___ Python tracker <http://bugs.python.org/issue5689> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5689] Support xz compression in tarfile module
Nadeem Vawda added the comment:
Yes, that's a good idea. I've been testing a similar change, and it seems
to drop the peak memory usage for test_tarfile from around 810MB down to
under 200MB. It looks like 2GB genuinely isn't enough to reliably use LZMA
compression with preset=9.
You might want to use preset=None instead of explicitly saying preset=6,
though. This tells LZMAFile to use the default preset, and will allow you
to get rid of the if-statement on lines 1821-1823.
Something unrelated that I noticed in the surrounding code: gzopen and
bz2open validate the mode by testing 'len(mode) > 1 or mode not in "rw"'.
This would be simpler as 'mode not in ("r", "w")' (like you've done in
xzopen), and it would accept only "r" and "w" (but not "" or "rw").
--
___
Python tracker
<http://bugs.python.org/issue5689>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5689] Support xz compression in tarfile module
Nadeem Vawda added the comment: Patch looks good to me. -- ___ Python tracker <http://bugs.python.org/issue5689> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Nadeem Vawda added the comment: Thanks for the patch. I'll review it tomorrow. -- ___ Python tracker <http://bugs.python.org/issue1625> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12167] test_packaging reference leak
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12167> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Nadeem Vawda added the comment: OK, I'll hold off on doing a detailed review until then. -- ___ Python tracker <http://bugs.python.org/issue1625> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Nadeem Vawda added the comment: > False alarm; go ahead with the review. I took a look too early in the > morning before caffeine kicked in. No worries. I know the feeling. The tests look fine. The bodies of testRead() and testReadMultiStream() appear to have been swapped, though. I'm guessing testRead() was supposed to remain unmodified, with testReadMultiStream() testing the case of streams=5? For the change to BZ2File._fill_buffer(), I'm not sure that the check for end-of-file is correct. It seems that if the end of rawblock lines up with the end of a stream, the mode will be set to EOF even if there are more streams waiting to be read. Is this possible? > A note on bz2 behavior: A BZ2Decompressor object is only good for one > stream; after that eof is set and it will refuse to continue to the > next stream; this seems in line with bzip2 manual I think this is the right way to do things. BZ2Decompressor is a low- level wrapper around the underlying C API -- it should not grow extra features unnecessarily. Also, certain use-cases might depend on being able to detect the end of a logical stream; this would not be possible if BZ2Decompressor were to be changed. The difference in behaviour from BZ2File and decompress() should probably be documented, though. -- ___ Python tracker <http://bugs.python.org/issue1625> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12190] intern filenames in bytecode
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12190> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Nadeem Vawda added the comment: Committed. Once again, thanks for the patch! -- resolution: -> accepted stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue1625> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Nadeem Vawda added the comment: I seem to be unable to log in to rietveld, so I'll reply here. >> result += decomp.decompress(data) > Is this efficient? I understood that other Python implementations > had poorly performing str.__iadd__, and therefore that using a list > was the common idiom (using “return b''.join(result)” at the end). Good point. I hadn't thought about other implementations. Also, you're right about the superfluous comments in test_bz2; I'll do a general cleanup of the test code soon. > Looks good. I only have one paranoid comment: since the tests use > self.TEXT * 5 to create multiple streams, the ordering of the files is > not tested. IOW, the code could mix-up the order of the files and the > tests would not catch that. Is it a concern? I wouldn't think so. It's not as though there is an index that the code looks at to find the location of each stream. It just reads the data from the file and if it reaches the end of one stream, it assumes that the following data is a new stream, and decompresses it accordingly. That said, I wouldn't be opposed to adding a test for that sort of thing (just for paranoia's sake :P) if it doesn't involve adding large amounts of additional binary data in the file. I'll come back to it once I've tidied up the existing code. -- ___ Python tracker <http://bugs.python.org/issue1625> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Nadeem Vawda added the comment: > If you’re logged into Roundup, you should automatically be logged into > our Rietveld instance. I thought this was the case, but it isn't working for me. I've filed a bug on the meta-tracker. -- ___ Python tracker <http://bugs.python.org/issue1625> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11802] filecmp.cmp needs a documented way to clear cache
Nadeem Vawda added the comment: > Also, I want to take a fresh look at the cache strategy (saving diffs > of two files vs saving file contents individually) and think about > whether than makes any sense at all for real world use cases > (is there a common need to compare the same file pairs over and over > again or is the typical use the comparison of many different file > pairs). There may even be a better way to approach the underlying > problem using hashes of entire files (md5, sha1, etc). I like that idea. A hash-based approach could speed up the detection of non-equal files quite a bit. -- ___ Python tracker <http://bugs.python.org/issue11802> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12304] expose signalfd(2) in the signal module
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12304> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12306] zlib: Expose zlibVersion to query runtime version of zlib
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12306> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12309] os.environ was modified by test_packaging
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12309> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10883] urllib: socket is not closed explicitly
Nadeem Vawda added the comment: Yes, the fix I provided only eliminated some of the warnings. As of fd6446a88fe3, test_urllib2net still leaks 5 sockets. -- ___ Python tracker <http://bugs.python.org/issue10883> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10883] urllib: socket is not closed explicitly
Nadeem Vawda added the comment: Here's an updated patch implementing reference counting for ftpwrapper. It changes the semantics of ftpwrapper.close() to postpone actually closing the connection until all files have also been closed (like socket.close()). -- Added file: http://bugs.python.org/file22547/issue10883.patch ___ Python tracker <http://bugs.python.org/issue10883> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10883] urllib: socket is not closed explicitly
Changes by Nadeem Vawda : -- resolution: -> accepted stage: -> patch review ___ Python tracker <http://bugs.python.org/issue10883> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10883] urllib: socket is not closed explicitly
Nadeem Vawda added the comment: The failure seems to occur sporadically. I'm looking into it. -- ___ Python tracker <http://bugs.python.org/issue10883> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10883] urllib: socket is not closed explicitly
Nadeem Vawda added the comment: The problem seems to be that CacheFTPHandler inherits ftp_open() from FTPHandler - FTPHandler.ftp_open() marks the ftpwrapper object to be closed as soon as the current transfer is complete. So CacheFTPHandler's cache ends up full of closed ftpwrappers. I don't have time to put together a solution now, but I'll work on something over the weekend. Another thing: CacheFTPHandler.clear_cache() sometimes breaks the cache, because it fails to clear self.timeout. Is there any reason why the timeouts need to be in a separate dict from the cached connections themselves? It seems like a very ugly and error-prone way of organizing things. -- assignee: -> nadeem.vawda ___ Python tracker <http://bugs.python.org/issue10883> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10883] urllib: socket is not closed explicitly
Nadeem Vawda added the comment: Updated patch with fixed refcounting mechanism. Also fixes clear_cache() in CacheFTPWrapper to leave the cache in a consistent state for subsequent use. -- Added file: http://bugs.python.org/file22609/issue10883-v2.patch ___ Python tracker <http://bugs.python.org/issue10883> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12555] PEP 3151 implementation
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12555> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12576] urlib.request fails to open some sites
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12576> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10883] urllib: socket is not closed explicitly
Changes by Nadeem Vawda : -- resolution: accepted -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker <http://bugs.python.org/issue10883> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12646] zlib.Decompress.decompress/flush do not raise any exceptions when given truncated input streams
Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker <http://bugs.python.org/issue12646> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
