[issue10050] urllib.request still has old 2.x urllib primitives

Changes by Senthil Kumaran : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue10050] urllib.request still has old 2.x urllib primitives

Roundup Robot added the comment: New changeset ea76cfff5851 by Senthil Kumaran in branch '3.3': #10050 - Document DeprecationWarnings for URLopener and FancyURLopener (msg172874 ) http://hg.python.org/cpython/rev/ea76cfff5851 New changeset d4cbd9e2e1cb by Senthil Kumaran in branch 'default': #1

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: I think, this issue should have been closed. Or at least be closed now. 1. urlretrieve - change was done. 2. Re: msg172874 - DeprecationWarnings on URLopener have been in place since 3.3. Also in the documentation in the Legacy Interface section, it is mentio

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: Ah, disregard. I followed up on the other bug. The legacy interface indeed should have stayed consistant. -- ___ Python tracker ___ __

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: Reverting of the len(block) back to 'bs' here aside, does it even make sense to include block information at all? That's the attempted read size, so it might not be an accurate representation of the size of the data actually read. Thus the reason for the initia

[issue10050] urllib.request still has old 2.x urllib primitives

Roundup Robot added the comment: New changeset e19a6194aee4 by Gregory P. Smith in branch '3.3': Fix test_urllib broken by my previous commits. The assumptions it was http://hg.python.org/cpython/rev/e19a6194aee4 New changeset dcf9a07830a6 by Gregory P. Smith in branch 'default': Fix test_urlli

[issue10050] urllib.request still has old 2.x urllib primitives

Changes by anatoly techtonik : -- versions: +Python 3.4 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue10050] urllib.request still has old 2.x urllib primitives

anatoly techtonik added the comment: Please note that changes to urlretrieve to return block size 0 in callback breaks existing PyPI apps, such as http://pypi.python.org/pypi/wget It would be nice if you revert this part from http://hg.python.org/cpython/rev/53715804dc71 Also if you change se

[issue10050] urllib.request still has old 2.x urllib primitives

akira added the comment: Related issue 16409 -- nosy: +akira ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue10050] urllib.request still has old 2.x urllib primitives

Ezio Melotti added the comment: See also #12707. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue10050] urllib.request still has old 2.x urllib primitives

Nadeem Vawda added the comment: Hmm, OK. URLopener and FancyURLopener do each issue a DeprecationWarning when used, though. If they are not actually deprecated, perhaps we should remove the warnings for the moment? -- ___ Python tracker

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: Only the classes which are marked as deprecated are allowed removed in the next release. So the ones which we marked as deprecated in 3.3 can safely go in 3.4. URLopener and FancyURLopener AFAIR had some dependency/ usage, deprecating those may require some refa

[issue10050] urllib.request still has old 2.x urllib primitives

Nadeem Vawda added the comment: Are we still planning on removing URLopener and FancyURLopener in 3.4? The documentation for 3.3 does not list these classes as deprecated. -- ___ Python tracker ___

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: Disregard. That was in concert with ntpath, which uses a funky approach to testing. -- ___ Python tracker ___ ___

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: I was looking at a somewhat unrelated issue and I bumped up against a similar situation with the warnings system. I didn't look too much into it, but it appeared that warnings didn't get added to __warningregistry__ correctly. Though, when I set the stack level

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: It's funny, Antoine had raised an issue too. Should see why is this sporadic failures.. -- ___ Python tracker ___ __

[issue10050] urllib.request still has old 2.x urllib primitives

Charles-François Natali added the comment: Since eab274c7d456, all the buildbots are randomly failing, e.g. """ == FAIL: test_method_deprecations (test.test_urllib2.OpenerDirectorTests) -

[issue10050] urllib.request still has old 2.x urllib primitives

Roundup Robot added the comment: New changeset 30f13d7fecd0 by Senthil Kumaran in branch 'default': Fix the buildbot breakdown - issue 10050 http://hg.python.org/cpython/rev/30f13d7fecd0 -- ___ Python tracker

[issue10050] urllib.request still has old 2.x urllib primitives

Roundup Robot added the comment: New changeset eab274c7d456 by Senthil Kumaran in branch 'default': deprecated the old urllib primitives in 3.3 urllib package - issue 10050 http://hg.python.org/cpython/rev/eab274c7d456 -- ___ Python tracker

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: My concern was with the method get_full_url. I have seen it being useful in more than one place and it's not just the accessor. Others are just the accessors and those can be safely removed. When get_full_url gives the correct url, taking into consideration the

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: I don't see why we'd need to make these _private -- they're just accessors/mutators for the most part. I'd be happy to help clean this up if you need it. -- ___ Python tracker ___

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: Here is patch which adds DeprecationWarning to URLOpener and (it's subclasses FancyURLopener) as well as some other Request methods. The idea would be, in the next release we remove this old URLOpener and it's subclasses and remove (or make _private) the Requ

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: Yes, it is simply a matter of adding warnings.warn in proper classes and methods. I have started on that, Jeff. No problem. I just wanted the changes to be two separate commits. Thanks! -- ___ Python tracker

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: I'd be happy to do that (URLopener, to begin with), though I'm not familiar with the usual approach. Is it simply a matter of warning in __init__? -- nosy: +mcjeff ___ Python tracker

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: One at the time - urlretrieve is re-written. Now should add deprecation warnings to things methods to be deprecated / removed. -- ___ Python tracker

[issue10050] urllib.request still has old 2.x urllib primitives

Roundup Robot added the comment: New changeset 53715804dc71 by Senthil Kumaran in branch 'default': Issue10050 - urlretrieve uses newer urlopen. reporthook of urlretrieve takes, block number, block read size, file_size http://hg.python.org/cpython/rev/53715804dc71 -- nosy: +python-dev

[issue10050] urllib.request still has old 2.x urllib primitives

Changes by Jeff McNeil : -- nosy: -mcjeff ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue10050] urllib.request still has old 2.x urllib primitives

Changes by Ezio Melotti : -- nosy: +ezio.melotti ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue10050] urllib.request still has old 2.x urllib primitives

Éric Araujo added the comment: Feel free to create them and set orsenthil as assignee. -- ___ Python tracker ___ ___ Python-bugs-list

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: I'd be happy to pick some of that stuff up. I'd like to address separately as it keeps fewer concerns in this one patch. I'll grab them once they're created. -- ___ Python tracker _

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: No, it is not waiting for you, in fact it is waiting for me to commit the portion which you have already contributed with some additional code (DeprecationWarning). But this bug does not end up here as there are other primitives which may need cleanup and ha

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: Just wanted to check so this doesn't sit with people waiting on me. Is there anything else I need/should do to this patch? Little unclear on how to handle the deprecation process. -- ___ Python tracker

[issue10050] urllib.request still has old 2.x urllib primitives

Éric Araujo added the comment: Antoine: you’re right about not mixing concerns, sorry. Regarding deprecation, we should be cautious and maybe plan it across more than two versions. See thread starting at http://mail.python.org/pipermail/python-dev/2011-March/109450.html --

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: I'm not exactly sure what the steps are with respect to the DeprecationWarning. Is the common case just to raise the warning in the __init__ method? Are there related documentation changes? Thanks again! Learning a ton. Hopefully the next patch I submit will go

[issue10050] urllib.request still has old 2.x urllib primitives

Antoine Pitrou added the comment: > URLopener which is an old class from the merge of urllib and urllib2 > and it can be slowly and safely removed. If we go this line, then I > assume it has to have a DeprecationWarning before we remove it. Should > we or not? Yes, we should. > Change URLopene

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: Antoine Pitrou wrote: > Can you clarify the issue? URLopener which is an old class from the merge of urllib and urllib2 and it can be slowly and safely removed. If we go this line, then I assume it has to have a DeprecationWarning before we remove it. Should w

[issue10050] urllib.request still has old 2.x urllib primitives

Antoine Pitrou added the comment: > There is one comment which I forgot to mention earlier. > > The current urlretrieve function is internally calling the URLOpener's > retrieve method. > > Those URLOpener class might need a DeprecationWarning while address > thing bug and that particular ret

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: Thanks again. Agree to use of returning the block size to the returnhook callable. But this is going be a deviation from the existing behavior which was present to inform the user at regular intervals and did not bother to provide any 'progressive size' relate

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: Take four! Includes Antoine's suggestions. I changed the callback to return (block num, read size, file size) as opposed to (block num, block size, file size) as this seems to make more sense. I appreciate the back and forth. I'd be happy to create issues & han

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: I'll make those changes, sure. I had the same thought re: block size, but I was trying to keep inline with what the current function did. -- ___ Python tracker

[issue10050] urllib.request still has old 2.x urllib primitives

Antoine Pitrou added the comment: I don't think the tests should be moved from one file to the other. It's making more difficult to tell whether you have changed them or not. I think moving the tests (as well as changing the synopsis, hello Eric) are cosmetic changes that are better done in s

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: Made requested change to Synopsis/Description. -- Added file: http://bugs.python.org/file21287/issue10050.patch ___ Python tracker ___ ___

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: Changing the description is a minor update. The term 'URL access module' seems fine. -- ___ Python tracker ___ __

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: Made recommended changes. Moved to NamedTemporaryFile. I don't think the spooled file makes sense here as the existing protocol provides a filename in the returned tuple, not a f.l.o. As far as the description? Here are a couple suggestions: 1. URL Retrieval

[issue10050] urllib.request still has old 2.x urllib primitives

Senthil Kumaran added the comment: Significant patch. Thanks. I looked at the review too. For the Context Manager part, don't club it along with this one. We need to test this thoroughly, after this is in shape, that can be addressed. -- ___ Python t

[issue10050] urllib.request still has old 2.x urllib primitives

Éric Araujo added the comment: Follow the “review” link next to the patch for an initial review. -- ___ Python tracker ___ ___ Python

[issue10050] urllib.request still has old 2.x urllib primitives

Jeff McNeil added the comment: Alright, attaching a patch that reworks urlretrieve to use urlopen internal to urllib.request. 1. I dropped the local caching as it isn't turned on by default anyway (and isn't really documented). 2. Updated documentation to reflect caching changes & make url

[issue10050] urllib.request still has old 2.x urllib primitives

Changes by Nadeem Vawda : -- nosy: +nvawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue10050] urllib.request still has old 2.x urllib primitives

Changes by Jeff McNeil : -- nosy: +mcjeff ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue10050] urllib.request still has old 2.x urllib primitives

Changes by Éric Araujo : -- nosy: +eric.araujo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue10050] urllib.request still has old 2.x urllib primitives

New submission from Antoine Pitrou : 3.x urllib.request still has a lot of stuff inherited from 2.x urllib: URLopener, FancyURLopener, the urlretrieve implementation (perhaps others?). Ideally, urlretrieve should be reimplemented using urlopen or build_opener, and the other stuff deprecated.