[issue15002] urllib2 does not download 4 MB file completely using ftp

Irit Katriel added the comment: 2.7 backport is no longer relevant. -- nosy: +iritkatriel status: open -> closed ___ Python tracker ___ ___

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: This is fixed in 3.4 and 3.5. I will backport to 2.7 ( I think, it is worth it). -- resolution: -> fixed stage: patch review -> resolved ___ Python tracker __

[issue15002] urllib2 does not download 4 MB file completely using ftp

Roundup Robot added the comment: New changeset bb71b71322a3 by Senthil Kumaran in branch '3.4': urllib.response object to use _TemporaryFileWrapper (and _TemporaryFileCloser) http://hg.python.org/cpython/rev/bb71b71322a3 New changeset 72fe23edfec6 by Senthil Kumaran in branch '3.4': NEWS entry f

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: Christian's patch is good.It helps in setting the socket.makefile file descriptor to a well behaving well file close wrapper and thus will help us prevent some tricky fd close issues. I added tests for coverage to ensure that we are asserting the type and co

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: Antoine, could you check my last comment in here? (The nosy list got reset accidentally when I made that comment and got a conflict from the tracker). -- nosy: +pitrou ___ Python tracker

[issue15002] urllib2 does not download 4 MB file completely using ftp

Antoine Pitrou added the comment: Well, this looks ok on the principle, but I haven't investigated the urllib-specific parts, so I'll let Senthil delve into this :) -- ___ Python tracker __

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Christian Theune : -- keywords: +patch Added file: http://bugs.python.org/file34915/d3c6ab639306.diff ___ Python tracker ___ __

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: Christian , with respect to patch, I agree with the logic (using something similar to #18879). Does all current unittests succeed with this? (I suspect not) A unittest for coverage would be helpful. -- ___ Python t

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: Antoine, I'm adding you here as I'm leveraging your patch from #18879. I'd need some feedback about the backport, but this patch should be OK for 3.4. Also, if you had an idea how to test this - I tried, but failed so far. -- __

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Thomas Wouters : -- stage: needs patch -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: I wasn't able to come up with a good testcase. :( I tried similar approaches as in #18879 but I wasn't able to make them trigger the behaviour as it also seems to be an issue regarding actual network performance ... :/ Backport to 2.7 is currently missing a

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Christian Theune : -- hgrepos: +237 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyt

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Christian Theune : -- versions: +Python 3.4, Python 3.5 -Python 3.2, Python 3.3 ___ Python tracker ___ ___ Python-bugs-list

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: This is actually the same problem as #18879. Changing the sample to keep a reference to the addinfourl object avoids this issue. This is even worse than #18879 in the sense that the error goes undetected and just leaves you with partial data. Looking at th

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: > I'll make another wireshark recording to get more details for me to analyze. Thank you! That will be useful. Please test it against 3.x version as it has seen cleanups recently. -- ___ Python tracker

[issue15002] urllib2 does not download 4 MB file completely using ftp

Christian Theune added the comment: Looking into this. It seems that it doesn't happen for all servers, I can download large files reliably from other sources. I'll make another wireshark recording to get more details for me to analyze. -- nosy: +ctheune

[issue15002] urllib2 does not download 4 MB file completely using ftp

Brian Brazil added the comment: More particularly, the ftpwrapper's ftp member is being GCed sometime after FtpHandler.ftp_open returns. -- ___ Python tracker ___ _

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by Antoine Pitrou : -- nosy: +giampaolo.rodola ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://m

[issue15002] urllib2 does not download 4 MB file completely using ftp

Brian Brazil added the comment: I've tested this on head, and the issue appears to be buggy ftp code in python. >From the attached tcpdump for fetching delegated-ripencc-20120706: 12:57:19.933607 IP myhost.39627 > ftp.ripe.net.ftp: Flags [.], ack 511, win 115, options [nop,nop,TS val 12935319

[issue15002] urllib2 does not download 4 MB file completely using ftp

Senthil Kumaran added the comment: That's surprising! I shall test it with http debug mode and see what's happening. -- assignee: -> orsenthil ___ Python tracker ___ _

[issue15002] urllib2 does not download 4 MB file completely using ftp

R. David Murray added the comment: The same problem exists in Python3. -- nosy: +orsenthil, r.david.murray stage: -> needs patch versions: +Python 3.2, Python 3.3 ___ Python tracker __

[issue15002] urllib2 does not download 4 MB file completely using ftp

Changes by sspapilin : -- title: urllib2 does not download 4 MB file fully using ftp -> urllib2 does not download 4 MB file completely using ftp ___ Python tracker ___ _