[Python-Dev] Is file.readlines(sizehint=N) broken on 2.7?
with open('test-xxx', 'w') as f: f.write('aaa\nbbb\nccc') with open('test-xxx', 'r') as f: print(f.readlines(1)) On Python 3.3 I get: ['aaa\n'] ...while on Python 2.7: ['aaa\n', 'bbb\n', 'ccc'] Is this a bug or I'm missing something? --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is file.readlines(sizehint=N) broken on 2.7?
2013/4/5 INADA Naoki : > The builtin open() was replaced with io.open(). > It's difference between file.readlines() and io.IOBase.readlines(). Should that justify this difference in behavior? Apparently on 2.X sizehint does not have any effect as far as I can see. --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is file.readlines(sizehint=N) broken on 2.7?
2013/4/5 R. David Murray : > On Fri, 05 Apr 2013 20:24:43 +0200, =?ISO-8859-1?Q?Giampaolo_Rodol=E0?= > wrote: >> 2013/4/5 INADA Naoki : >> > The builtin open() was replaced with io.open(). >> > It's difference between file.readlines() and io.IOBase.readlines(). >> >> Should that justify this difference in behavior? > > Yes. The 'file' documentation for readlines says documents that the > argument is advisory and may be rounded up to an internal buffer size. > > io's readlines, on the other hand, says no more lines will be read if > the total size of bytes read so far exceeds the hint. > > So, different semantics. I see. >> Apparently on 2.X sizehint does not have any effect as far as I can see. > > Have you played with large enough hints/small enough buffers? Right, it seems it tends to change around 8196 bytes. Ok then, nevermind. --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The end of 2.7
2013/4/8 Stephen Hansen : > On Sun, Apr 7, 2013 at 6:53 AM, Christian Tismer > wrote: >> >> On 07.04.13 14:10, Skip Montanaro wrote: >> >> Where I work (a trading firm that uses Python as just one of many >> different pieces of technology, not a company where Python is the core >> technology upon which the firm is based) we are only just now >> migrating from 2.4 to 2.7. I can't imagine we'll have migrated to >> Python 3 in two years. It's not like we haven't seen this coming, but >> you can only justify moving so fast with technology that already >> works, especially if, like Python, you use it with lots of other >> packages (most/all of which themselves have to be ported to Python 3) >> and in-house software. >> >> I think the discussion should focus on who's left on 2.x and why, not, >> "yeah, releases every six months for the next couple years ought to do >> it." >> >> >> when I read this, I was slightly shocked. You know what? >> """ >> We are pleased to announce the release of Python 2.4, final on November >> 30, 2004. >> """ >> >> I know that companies try to save (time? money?) something by not >> upgrading >> software, and this is extremely annoying. > > > I'm in the same boat as Skip (just now moving from 2.4 to 2.7), and Python > *is* a core technology for us. It has nothing really to do with saving time > or money, its about priorities. The transition from 2.3 to 2.4 was actually > fairly painful (don't ask me why, I don't even remember anymore), but we got > stuck on 2.4 not by any specific decision -- it simply worked, and our time > was always focused upon solving problems and improving our software itself. > > Could we have solved our problems easier if we upgraded Python and had new > tools? Some, yes. (Some features we have added had me actually walking > through third party code bases and backporting it -- converting with to > try/finally is an amusing big one for example) > > For one thing, even with this relatively ancient Python, we almost never ran > into bugs. It just worked and worked fine, so when we looked at our > development plan the list of feature requests and issues for various > customers (especially those that were potential new clients) overrode > "infrastructure" upgrades as priorities. > > However, in a huge system that has many tens of thousands of lines of code, > doing a platform upgrade is just a serious endeavor -- and its often not > even Python's fault itself, but the reality that it means we're going to be > upgrading *everything* and involves a much more involved QA cycle and often > runs into third party software. We are finally upgrading now because the > time to work around certain bugs in both Python and third-party libraries > that no longer support 2.4 are enough for us to say, okay, we finally really > do need to get this done. > > Migration to Python 3 ... IF it ever happens is more of a question then > when. > > That's not a indictment of Python 3 or a problem with the current plan (for > what its worth, the bugfix every 6 months until 5 years is up seems totally > reasonable). > > Any new product we do, I'd seriously consider starting from Python 3. > (Though PyPy supporting Py3 would help that argument a lot) The case for > migrating existing products is a lot harder to make. You might also think about rewriting the code so that it "kind of" works on both 2.7 *and* 3.3. By that I mean that your code on Python 3 should not necessarily work but neither it should raise SyntaxError. If from the start you use: - six - "except exception as:" - "__future__ module’s from __future__ import division, print_statement, unicode_literals" - fix warnings signaled by "python -3 app.py" ...and other similar tricks, your ported code is likely to look nicer and more modern and a future porting to Python 3 should be a lot less painful. At least, if the circumstances are right, I personally might see some value in doing so. --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Email addresses for new committers for python-committers
I'm already subscribed as "g.rod...@gmail.com" and I'm able to receive messages from the list. 2010/4/20 Brett Cannon : > If you are a committer and are NOT subscribed to the python-committers > mailing list (I believe this at least includes Giampaolo, JP, and Brian), > then please either reply to this email with your preferred email address or > let me know directly (the former is preferred so Georg or Eric can beat me > to the subscription if I take too long). > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Documenting [C]Python's Internals
2010/5/20 Yaniv Aknin : > Hi, > I wanted to let python-dev know about a series of articles about CPython's > internals I'm publishing under the collective title "Guido's Python"* > (http://tech.blog.aknin.name/tag/guidos-python/). Three articles already > were published already, more are planned (mainly focused on CPython/py3k, > but comparisons with other implementations may also be covered; we'll > see). So far I've done an introduction/whirlwind tour of Py_Main and a > two-article in-depth review of the (new-style) object system. > I'm sharing this with you (and hope you care) due to three reasons, probably > in escalating importance: > (a) Maybe some of python-dev's readers would be interested (possibly the > newer and more silent members). > (b) Maybe my scales are wrong, but I was a bit surprised by the number of > readers (>20,000 in the past two weeks); I wouldn't want to mislead such a > reader base and would be happy if a veteran here would be interested in > aiding by technically proofing the material (shan't be too hard I hope, feel > free to contact me directly if qualified and interested). > (c) While the content is currently geared to be blog-oriented, if it's found > worthy by the group I'd be delighted to formulate it into something more > 'reference-material-ish' and give it back to the community. I found no > centrally organized CPython-internals material other than bits and pieces > (descrintro, eclectic blog posts, lectures, C-API reference, etc), and I > hope maybe something like this could be featured more officially > on python.org, with the relevant 'this is subject to change' disclaimers > (can be a document for new contributors, for pure Python programmers who're > just interested, or for whatever we decide). > Questions? Comments? > - Yaniv > * think "Tim Berners-Lee's Web" or "Keanu Reeves' Green Gibberish", see the > first post for details > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com Great! This can be *extremely* useful for new developers like me who still haven't took a look at cPython internals. Thanks for the effort. --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Reasons behind misleading TypeError message when passing the wrong number of arguments to a method
>>> class A: ... def echo(self, x): ... return x ... >>> a = A() >>> a.echo() Traceback (most recent call last): File "", line 1, in TypeError: echo() takes exactly 2 arguments (1 given) >>> I bet my last 2 cents this has already been raised in past but I want to give it a try and revamp the subject anyway. Is there a reason why the error shouldn't be adjusted to state that *1* argument is actually required instead of 2? --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reasons behind misleading TypeError message when passing the wrong number of arguments to a method
2010/5/20 John Arbash Meinel : > Giampaolo Rodolà wrote: >>>>> class A: >> ... def echo(self, x): >> ... return x >> ... >>>>> a = A() >>>>> a.echo() >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: echo() takes exactly 2 arguments (1 given) >> >> I bet my last 2 cents this has already been raised in past but I want >> to give it a try and revamp the subject anyway. >> Is there a reason why the error shouldn't be adjusted to state that >> *1* argument is actually required instead of 2? >> > > Because you wouldn't want to have > > A.echo() > > Say that it takes 1 argument and (-1 given) ? > > John > =:-> > > I see that as a different error type: what you're doing there is calling a method of a class which you haven't instantiated in the first place. Actually the error message returned in this other case is not very clear as well: "unbound method echo() must be called with A instance as first argument (got nothing instead)" It talks about "arguments" while no arguments are actually involved in the problem: just a class I forgot to initialize. --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] email package status in 3.X
2010/6/17 Bill Janssen : > There's a related meta-issue having to do with antique protocols. Can I know what meta-issue are you talking about exactly? > FTP, for instance, was designed when the Internet had only 19 nodes connected > together with custom-built refrigerator-sized routers. A very early > experiment in application protocols. It does a few odd things that > we've since learned to be inefficient/unwise/unnecessary. Does it make > sense that Python support every part of it? Being FTP protocol still quite widespread I'd say it makes a lot of sense. That aside, what parts of urllib/http* are penalized because of FTP support? --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] email package status in 3.X
2010/6/18 Bill Janssen : > Giampaolo Rodolà wrote: > >> 2010/6/17 Bill Janssen : >> >> > There's a related meta-issue having to do with antique protocols. >> >> Can I know what meta-issue are you talking about exactly? > > Giampaolo, I believe that you and I have already discussed this on one > of the FTP issues. > > Bill I only remember a discussion in which I was against removing OOB data support from asyncore in order to support certain parts of the FTP protocol using it, but that's all. I don't see how urlib or any other stdlib module is supposed to be penalized by FTP protocol in any way. --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] email package status in 3.X
2010/6/20 Steven D'Aprano : > Python 2.x introduced Unicode strings. Python 3.x merely makes them the > default. "Merely"? To me this looks as the main reason why a lot of projects haven't been ported to Python 3 yet. I attempted to port pyftpdlib to python 3 several times and the biggest show stopper has always been the bytes / string difference introduced by Python 3 which forces you to *know* and *use* Unicode every time you deal with some text and 2to3 is completely useless here. I can only imagine how difficult can it be to do such a conversion in a project like Twisted or Django where the I/O plays a fundamental role. The choice of forcing the user to use Unicode and "think in Unicode" was a very brave one, and I'm sure it's for the better, but not everyone wants to deal with that because Unicode is hard to swallow. The majority of people prefer to stay with bytes and eventually learn and introduce Unicode only when that is actually needed. --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing IDLE from the standard library
One of the main problems with IDLE is the lack of tabs for editing multiple files within the same window. Having that alone would be a great improvement. --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing IDLE from the standard library
2010/7/14 Guilherme Polo : > 2010/7/14 Terry Reedy : >> On 7/14/2010 2:35 AM, Giampaolo Rodolà wrote: >>> >>> One of the main problems with IDLE is the lack of tabs for editing >>> multiple files within the same window. >>> Having that alone would be a great improvement. >> >> Yes, the same as tabs for browsing was. >> >> This is firstly an unlying gui widget set issue. Tk does not, as far as I >> know, have a tabbed document widget. Ttk has a new Notebook widget, with >> tabs. >> > > I have worked on this before, and I can tell you that simply changing > to ttk widgets is the easiest part. My recommendation, that you are > free to ignore (especially if you want to skip this previous work), > is: as a first step change the EditWindow to act more like a EditPage. > That is, you should be able to instantiate a EditWindow and include > this new EditPage on it (as a child or something else that you may > imagine). > >> -- >> Terry Jan Reedy >> > > > > -- > -- Guilherme H. Polo Goncalves > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > http://code.google.com/p/python-ttk/wiki/Screenshots Modified IDLE screenshots look impressive. Has Ttk (or something similar) ever been considered for inclusion? I think the problem here is more Tkinter itself rather than IDLE. Does spending energies on working on something (IDLE) based on such an old and features-limited module as Tkinter really worth the effort? Some major lacks like tabbed browsing can be fixed somehow but there always will be a big impediment deriving from using such an old graphic library which can't provide all the facilities offered by wx, gtk, etc... Maybe it would be better to first upgrade/improve Tkinter somehow; provide a gui toolkit that is *really* competitive, then start to seriously work on IDLE. My 2 cents --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing IDLE from the standard library
http://docs.python.org/dev/whatsnew/2.7.html#ttk-themed-widgets-for-tk Sorry, I realized just now that ttk is already included in Python 2.7 and 3.2. 2010/7/14 Giampaolo Rodolà : > 2010/7/14 Guilherme Polo : >> 2010/7/14 Terry Reedy : >>> On 7/14/2010 2:35 AM, Giampaolo Rodolà wrote: >>>> >>>> One of the main problems with IDLE is the lack of tabs for editing >>>> multiple files within the same window. >>>> Having that alone would be a great improvement. >>> >>> Yes, the same as tabs for browsing was. >>> >>> This is firstly an unlying gui widget set issue. Tk does not, as far as I >>> know, have a tabbed document widget. Ttk has a new Notebook widget, with >>> tabs. >>> >> >> I have worked on this before, and I can tell you that simply changing >> to ttk widgets is the easiest part. My recommendation, that you are >> free to ignore (especially if you want to skip this previous work), >> is: as a first step change the EditWindow to act more like a EditPage. >> That is, you should be able to instantiate a EditWindow and include >> this new EditPage on it (as a child or something else that you may >> imagine). >> >>> -- >>> Terry Jan Reedy >>> >> >> >> >> -- >> -- Guilherme H. Polo Goncalves >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com >> > > http://code.google.com/p/python-ttk/wiki/Screenshots > Modified IDLE screenshots look impressive. > Has Ttk (or something similar) ever been considered for inclusion? > I think the problem here is more Tkinter itself rather than IDLE. > Does spending energies on working on something (IDLE) based on such an > old and features-limited module as Tkinter really worth the effort? > > Some major lacks like tabbed browsing can be fixed somehow but there > always will be a big impediment deriving from using such an old > graphic library which can't provide all the facilities offered by wx, > gtk, etc... > > Maybe it would be better to first upgrade/improve Tkinter somehow; > provide a gui toolkit that is *really* competitive, then start to > seriously work on IDLE. > > > My 2 cents > > --- Giampaolo > http://code.google.com/p/pyftpdlib > http://code.google.com/p/psutil > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] profiler decorator - is it worth for inclusion?
Today I was looking for a quick and dirty way to profile a method of a class. I was thinking that cProfile module had a decorator for this but I was wrong so I decided to write one based on hotshot. Would it be worth for inclusion? #!/usr/bin/env python import hotshot import hotshot.stats import tempfile import pstats def profile(sort='cumulative', lines=30, strip_dirs=False): """A decorator which profiles a callable. Example usage: >>> @profile() ... def factorial(n): ... n = abs(int(n)) ... if n < 1: ... n = 1 ... x = 1 ... for i in range(1, n+1): ... x = i * x ... return x ... >>> factorial(5) 1 function calls in 0.000 CPU seconds Ordered by: internal time, call count ncalls tottime percall cumtime percall filename:lineno(function) 10.0000.0000.0000.000 profiler.py:60(factorial) 00.000 0.000 profile:0(profiler) 120 >>> """ def outer(fun): def inner(*args, **kwargs): file = tempfile.NamedTemporaryFile() prof = hotshot.Profile(file.name) try: ret = prof.runcall(fun, *args, **kwargs) finally: prof.close() stats = hotshot.stats.load(file.name) if strip_dirs: stats.strip_dirs() if isinstance(sort, tuple): stats.sort_stats(*sort) else: stats.sort_stats(sort) stats.print_stats(lines) return ret return inner return outer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] profiler decorator - is it worth for inclusion?
2010/7/15 Brian Curtin : > On Thu, Jul 15, 2010 at 13:45, Giampaolo Rodolà wrote: >> >> Today I was looking for a quick and dirty way to profile a method of a >> class. >> I was thinking that cProfile module had a decorator for this but I was >> wrong so I decided to write one based on hotshot. >> Would it be worth for inclusion? > > Since hotshot is gone in 3.x, I'd guess the chances are probably slim. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > > Here's one using cProfile instead. I was using hotshot because I wasn't aware of cProfile.Profile.runcall which is currently not documented (I'm going to file a bug report). def profile(sort='cumulative', lines=30, strip_dirs=True): """A decorator which profiles a callable. Example usage: >>> @profile() ... def factorial(n): ... n = abs(int(n)) ... if n < 1: ... n = 1 ... x = 1 ... for i in range(1, n+1): ... x = i * x ... return x ... >>> factorial(5) Thu Jul 15 20:58:21 2010/tmp/tmpIDejr5 4 function calls in 0.000 CPU seconds Ordered by: internal time, call count ncalls tottime percall cumtime percall filename:lineno(function) 10.0000.0000.0000.000 profiler.py:120(factorial) 10.0000.0000.0000.000 {range} 10.0000.0000.0000.000 {abs} 10.0000.0000.0000.000 {method 'disable' of '_lsprof.Profiler' objects} 120 """ def outer(fun): def inner(*args, **kwargs): file = tempfile.NamedTemporaryFile() prof = cProfile.Profile() try: ret = prof.runcall(fun, *args, **kwargs) except: file.close() raise prof.dump_stats(file.name) stats = pstats.Stats(file.name) if strip_dirs: stats.strip_dirs() if isinstance(sort, tuple): stats.sort_stats(*sort) else: stats.sort_stats(sort) stats.print_stats(lines) file.close() return ret return inner return outer --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] profiler decorator - is it worth for inclusion?
Provided a patch on the tracker: http://bugs.python.org/issue9285 Further comments can be submitted there, if any. --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil 2010/7/15 Giampaolo Rodolà : > 2010/7/15 Brian Curtin : >> On Thu, Jul 15, 2010 at 13:45, Giampaolo Rodolà wrote: >>> >>> Today I was looking for a quick and dirty way to profile a method of a >>> class. >>> I was thinking that cProfile module had a decorator for this but I was >>> wrong so I decided to write one based on hotshot. >>> Would it be worth for inclusion? >> >> Since hotshot is gone in 3.x, I'd guess the chances are probably slim. >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com >> >> > > > Here's one using cProfile instead. > I was using hotshot because I wasn't aware of cProfile.Profile.runcall > which is currently not documented (I'm going to file a bug report). > > > def profile(sort='cumulative', lines=30, strip_dirs=True): > """A decorator which profiles a callable. > > Example usage: > > >>> @profile() > ... def factorial(n): > ... n = abs(int(n)) > ... if n < 1: > ... n = 1 > ... x = 1 > ... for i in range(1, n+1): > ... x = i * x > ... return x > ... > >>> factorial(5) > Thu Jul 15 20:58:21 2010 /tmp/tmpIDejr5 > > 4 function calls in 0.000 CPU seconds > > Ordered by: internal time, call count > > ncalls tottime percall cumtime percall filename:lineno(function) > 1 0.000 0.000 0.000 0.000 profiler.py:120(factorial) > 1 0.000 0.000 0.000 0.000 {range} > 1 0.000 0.000 0.000 0.000 {abs} > 1 0.000 0.000 0.000 0.000 {method 'disable' of > '_lsprof.Profiler' objects} > > 120 > """ > def outer(fun): > def inner(*args, **kwargs): > file = tempfile.NamedTemporaryFile() > prof = cProfile.Profile() > try: > ret = prof.runcall(fun, *args, **kwargs) > except: > file.close() > raise > > prof.dump_stats(file.name) > stats = pstats.Stats(file.name) > if strip_dirs: > stats.strip_dirs() > if isinstance(sort, tuple): > stats.sort_stats(*sort) > else: > stats.sort_stats(sort) > stats.print_stats(lines) > > file.close() > return ret > return inner > > return outer > > > --- Giampaolo > http://code.google.com/p/pyftpdlib > http://code.google.com/p/psutil > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r84355 - python/branches/py3k/Lib/test/test_ssl.py
Sorry, I didn't get how the context-manager actually worked. Fixed in r84356. 2010/8/29 Michael Foord : > On 30/08/2010 00:23, Antoine Pitrou wrote: >> >> On Sun, 29 Aug 2010 22:56:56 +0200 (CEST) >> giampaolo.rodola wrote: >>> >>> + with self.assertRaises(IOError) as err: >>> + ssl.wrap_socket(socket.socket(), certfile=WRONGCERT) >>> + self.assertEqual(err.errno, errno.ENOENT) >> >> The assertEqual will never get executed since the previous line raises. > > If it is dedented once then it will work (in Python 2.7 / 3.2). > > Michael >>> >>> + with self.assertRaises(IOError) as err: >>> + ssl.wrap_socket(socket.socket(), certfile=WRONGCERT, >>> keyfile=WRONGCERT) >>> + self.assertEqual(err.errno, errno.ENOENT) >> >> Same here. >> >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > > -- > http://www.ironpythoninaction.com/ > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Internal counter to debug leaking file descriptors
> Of course it would be nice to get access to FD stack so that a > full filename can also be retrieved in this case. On Linux, this can be easily achieved by using /proc. You can take a look at how this is done in the current development version of psutil: http://code.google.com/p/psutil/source/browse/trunk/psutil/_pslinux.py?spec=svn633&r=630#266 Usage: >>> import psutil, os >>> this_process = psutil.Process(os.getpid()) >>> f = open('file.ext', 'w') >>> this_process.get_open_files() ['/home/giampaolo/svn/psutil/file.ext'] Same for sockets, a bunch of lines later: http://code.google.com/p/psutil/source/browse/trunk/psutil/_pslinux.py?spec=svn633&r=630#284 >>> import socket >>> s = socket.create_connection(('google.com', 80)) >>> this_process.get_connections() [connection(family=2, type=1, local_address=('192.168.1.43', 38067), remote_address=('72.14.234.104', 80), status='ESTABLISHED')] >>> Hope this helps --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Internal counter to debug leaking file descriptors
The Windows part slipped under my radar. =) Unfortunately the Windows binaries still refer to the current version which doesn't include open files and open connections functionalities. To have those he'll have to get the latest code from svn and compile it with mingw32. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2010/9/3 Terry Reedy : > On 9/3/2010 6:09 AM, Giampaolo Rodolà wrote: >>> >>> Of course it would be nice to get access to FD stack so that a >>> full filename can also be retrieved in this case. >> >> On Linux, this can be easily achieved by using /proc. >> You can take a look at how this is done in the current development >> version of psutil: >> >> http://code.google.com/p/psutil/source/browse/trunk/psutil/_pslinux.py?spec=svn633&r=630#266 >> Usage: >> >>>>> import psutil, os >>>>> this_process = psutil.Process(os.getpid()) >>>>> f = open('file.ext', 'w') >>>>> this_process.get_open_files() >> >> ['/home/giampaolo/svn/psutil/file.ext'] >> >> Same for sockets, a bunch of lines later: >> >> http://code.google.com/p/psutil/source/browse/trunk/psutil/_pslinux.py?spec=svn633&r=630#284 >> >>>>> import socket >>>>> s = socket.create_connection(('google.com', 80)) >>>>> this_process.get_connections() >> >> [connection(family=2, type=1, local_address=('192.168.1.43', 38067), >> remote_address=('72.14.234.104', 80), status='ESTABLISHED')] > > If you can use psutil itself, it has compiled Windows versions for 2.7 and > 3.1 > https://code.google.com/p/psutil/ > > -- > Terry Jan Reedy > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rietveld integration into Roundup
Thanks for this. It looks very nice. 2010/10/2 "Martin v. Löwis" : > Following up to the recent thread, I have now integrated Rietveld > into Roundup. This is a rough draft still, and highly experimental. > Please try it out, but expect that it may be necessary to discard > all data (including comments) to start over (of course, I'll try > to avoid having to do so). > > The Rietveld integration currently works this way: > - the installation lives at http://bugs.python.org/review/ > - for each roundup user, a rietveld user is created; > log into roundup to get access to rietveld > - for each roundup issue, a rietveld issue is created with the > same issue id. Please don't create additional Rietveld issues. > - regularly (currently every hour), each patch is considered. > Patches are skipped if: > * they have been added to Rietveld already > * they have no clear base version (i.e. they don't originate > from svn diff) > * they belong to no or a closed issue > * they apply to a patch that is not currently supported > (only trunk, 2.6, 2.7, 3.1, py3k are currently supported) > - for each such patch, a patchset is created > - if that is successful, a "review" link is added to roundup > > Feel free to discuss this here; bug reports and feature requests > should go to the meta tracker. Contributions are welcome; > I won't be able to work on this much for the next four days. > > Regards, > Martin > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
2010/10/8 Eric Smith : > On 10/8/10 10:26 AM, Barry Warsaw wrote: >> In any case, these could be a simple shell script wrapping 'python -m >> setup'. >> It could even take a --use-python-version option to select the pythonX.Y >> it >> used, without having to encode the Python version number in the script >> name. > > On Windows it can't be a shell script or batch file, but needs to be an > executable. setuptools already deals with this. If that's the case what would I type in the command prompt in order to install a module? "C:\PythonXX\pysetup.exe"? If so I would strongly miss old "setup.py install". --- Giampaolo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
Wouldn't be kinda weird that one can open the command prompt and run "pysetup" but not "python" on Windows? I recall an old issue on the bug tracker in which the latter proposal was widely discussed and finally rejected for reasons I can't remember (and it seems I can't even find the bug right now). I think it's likely that those same reasons are valid for "pysetup" in the same manner. For the record, I would be more than happy to be able to open the command prompt and type "pysetup" and "python" with success, one day. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2010/10/12 Eric Smith : > On 10/11/2010 5:17 PM, Giampaolo Rodolà wrote: >> >> 2010/10/8 Eric Smith: >>> >>> On 10/8/10 10:26 AM, Barry Warsaw wrote: >>>> >>>> In any case, these could be a simple shell script wrapping 'python -m >>>> setup'. >>>> It could even take a --use-python-version option to select the pythonX.Y >>>> it >>>> used, without having to encode the Python version number in the script >>>> name. >>> >>> On Windows it can't be a shell script or batch file, but needs to be an >>> executable. setuptools already deals with this. >> >> If that's the case what would I type in the command prompt in order to >> install a module? >> "C:\PythonXX\pysetup.exe"? >> If so I would strongly miss old "setup.py install". > > Same thing you would type at a shell prompt. Presumably we're talking about > "pysetup install" (which you'll note is one character shorter!). You could > fully qualify the path if need be, on any platform, using its conventions. > > Eric. > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Distutils2 scripts
2010/10/12 Antoine Pitrou : > On Tue, 12 Oct 2010 01:11:24 +0200 > Giampaolo Rodolà wrote: >> Wouldn't be kinda weird that one can open the command prompt and run >> "pysetup" but not "python" on Windows? > > If you add C:\PythonXY to your path, you can run "python". I know. My point was you can't do it by default and installing a module is something even a less experienced user usually does. Typing "C:\PythonXX\pysetup" is harder compared to "setup.py install" and solving this problem by modifying your environment paths so that you can just type "pysetup" is something I would expect to be done by the MSI installer, not the user. --- Giampaolo http://code.google.com/p/pyft/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for async read/write
You should file a new issue on the bug tracker but unless you have a patch to propose it's unlikely that someone else is gonna implement it. Regards --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2010/10/19 Jesus Cea : > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Current Python lacks support for "aio_*" syscalls to do async IO. I > think this could be a nice addition for python 3.3. > > If you agree, I will create an issue in the tracker. If you think the > idea is of no value, please say so for me to move on. Maybe an 3th party > module, but I think this functionality sould be available in core python. > > Thanks!. > > PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read, > aio_return, aio_write. > > - -- > Jesus Cea Avion _/_/ _/_/_/ _/_/_/ > j...@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ > jabber / xmpp:j...@jabber.org _/_/ _/_/ _/_/_/_/_/ > . _/_/ _/_/ _/_/ _/_/ _/_/ > "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ > "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ > "El amor es poner tu felicidad en la felicidad de otro" - Leibniz > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQCVAwUBTL3MUplgi5GaxT1NAQKv5QQAnDan88kXJ67fucz2rT/mZze+065lm9E4 > +XJ2JfqGMVE1/qMsXwg81l19RHSYReBgBjd7zyXWE9Fk/1Rfq4gjOZEtoG0IpGZG > E3CH+Ab5/o/PjJZNKQaPpe0cwGSXFPKkPpgepKS1d8ZXyf6VXMc8UTSWjMI5jIVe > 4M+yvw5m4I0= > =nsdO > -END PGP SIGNATURE- > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
2010/10/29 Barry Warsaw : > I had a brief conversation with Michael Foord yesterday and he's writing code > that works in 2.4 through 3.2, so for *some* code bases, it's tricky and ugly, > but possible. If the application does not involve a lot of I/O, 2.4 -> 3.2 support by using a unique code base is possible and not that ugly. At least, that's what happened with psutil: http://code.google.com/p/psutil/issues/detail?id=75&can=1&q=python%203&colspec=ID%20Summary%20Type%20Opsys%20Status%20Milestone%20Opened%20Owner%20Progress#c9 My personal choice is to encourage the same approach where possible. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] SSH access against buildbot boxes
Hi, sorry in advance if this sounds a little indiscreet, but I think it would be great if we'd have SSH access against some of the computers used to host buildbots. Personally, I would find this particularly useful for OSX since it's one of the few OSes I can't manage to virtualize and which often causes me problems. Some examples: http://bugs.python.org/issue10340 http://bugs.python.org/issue8490 (this one also involves Solaris) In such cases I would find more easy to be able to connect to the machine and test myself rather than create a separate branch, commit, schedule a buildbot run, wait for it to complete and see whether everything is "green". On the other side I perfectly understand how opening up blanket ssh access is not something everyone is comfortable with doing. AFAICR there was someone who was setting up an evironment to solve exactly this problem but I'm not sure whether this is already usable. Best regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
+1 on everything. 2010/11/11 Alexander Belopolsky : > 2010/11/11 Michael Foord : > .. >>> You mean runtime automation, e.g. creating __all__ on the fly omitting >>> underscored names? >>> >> Writing code to generate a __all__ that duplicates the default behaviour >> seems redundant to me. >> > > FWIW, I like having __all__ at the top of the module. It feels like a > table of contents at the start of a chapter. In some cases it may > also serve as an optimization when len(__all__) is much smaller than > len(__dict__). I also don't like _ prefix to become an exclusive > means to express privateness. > > I think the current definition of "public names" is a good one and > just needs to be made more visible in the docs. If the module defines > __all__, that should be the ultimate answer to what is public in that > module. (Users should learn to use help(module) instead of > dir(module) for API discovery.) If __all__ is not defined in the > module, I think it is good to introduce it after a careful review of > what it should contain. And __all__ should never contain names that > start with _. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] os.getpriority() and os.setpriority()
Hi all, after having implemented something very similar in psutil: http://code.google.com/p/psutil/issues/detail?id=142 ...I decided to contribute a patch for exposing getpriority() and setpriority() system calls in the os module: http://bugs.python.org/issue10784 This was also raised some time ago: http://mail.python.org/pipermail/python-ideas/2009-April/004028.html ...and it seems feedbacks were positive. I think it would be nice to have Windows equivalents (GetPriorityClass / SetPriorityClass) as well but I'm not sure how since their notation is different. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API refactoring tracker field for Python4
> Module split: > try to get all issues for 'os' module > try to subscribe to all commits for 'CGIHTTPServer' +1 I've been thinking about such a thing as well and I think it would be useful. Every now and then I go to the bug tracker to see whether the modules I usually maintain (mainly ftplib, asyncore, asynchat) need some attention. I do this by using the plain text search but a select box containing all the module names would be better. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/1/7 anatoly techtonik : > On Fri, Jan 7, 2011 at 7:41 PM, Brian Curtin wrote: >>> >>> There are many API changes and proposals that were forgotten and >>> didn't get into Python 3, although they should be, because it was the >>> only chance to change things with backwards compatibility break. For >>> example http://bugs.python.org/issue1559549 >> >> That can be added in 3.3. >> To answer your comment on the issue: no investigation is needed. It didn't >> make it in yet because there was no code written for it. It's really not a >> big deal, it happens all the time. > > Don't you think that if more people were aware of this issue, the > patch could be made faster? > >>> This happened, because of poor bug management, where community doesn't >>> play any role in determining which issues are desired. >> >> The community absolutely plays a role in determining which issues are >> desired. They do this by action when they want something. A patch says a >> whole lot about desire. >> > Don't you think that if people could review issues and "star" them > then such minor issues could be scheduled for release not only by > "severity" status as decided be release manager and several core > developers, but also by community vote? > > Patch requires time, experience and approved contribution agreement, > which you've sent using ground mail beforehand. Voting doesn't require > any of this, but helps core developers see what user community wants. > With the list of desired features Jesse Noller sponsored sprints will > have more value for all of us. > >>> >>> This mostly because of limitation of our tracker and desire of people >>> to extend it to get damn "stars", module split, sorting, digging and >>> tagging options. >> >> I have no idea what any of this means. > > Stars: > go http://code.google.com/p/support/issues/list > find Stars column > guess > > Module split: > try to get all issues for 'os' module > try to subscribe to all commits for 'CGIHTTPServer' > > Sorting: > click on column titles in bug tracker search results > > Tagging: > as a tracker user, try to add tag 'easy' to some easy issue > >>> >>> I won't be surprised if things won't change in the next couple of >>> years, that's why I'd like to propose a very small change, so that >>> when time will come to create Python4 (and standard library won't be >>> separated from interpreter by this time), everybody can get quickly >>> get a list of proposed API enhancements and filter which are eligible >>> for the next BC API break. This change is a simple "api-refactoring" >>> flag that could be added to corresponding issues by tracker users. >> >> I'm not sure I see the need for such a flag, as there are probably too few >> cases for this in the first place. > > I haven't started using Python 3 yet, but I already know some annoying > API issues that are not fixed there. Unfortunately, I don't remember > them to give you a list. That's why I asked for a flag. > -- > anatoly t. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add sendfile() to core?
A strong +1. Projects such as Twisted would certainly benefit from such an addiction. I'm not sure the os module is the right place for sendfile() to land though. Implementation between different platforms tends to vary quite a bit. A good resource is the samba source code which contains an implementation for all major UNIX systems. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/1/8 max ulidtko : > On Wed, 20 Mar 2002 14:53:58 -0500, Andrew Kuchling wrote: > | sendfile() is used when writing really high-performance Web servers, > | in order to save an unnecessary memory-to-memory copy. Question: > | should I make up a patch to add a sendfile() wrapper to Python? > > So, was this proposal rejected? If so, for what reasons? > > Wrapper of such a useful call would be of great convenience, especially > considering its availability on Win32 (see Thomas Heller's notice, [1]). > > Anyway, when one needs to send (arbitrarily large) files to a socket and > back, ugly and slow workarounds are born, like this one: > > def copy_file(file1, file2, length, blocksize=40960): > """ Transfer exactly length bytes from one file-like object to > another """ > sofar = 0 > while sofar < length: > amount = blocksize if sofar + blocksize <= length \ > else length - sofar > file2.write(file1.read(amount)) > sofar += amount > > > Using hypothetical os.sendfile() would be so much better! > > The only difficulty I can see is the choice of name for the wrapper. > IMO, using "sendfile" from Linux and FreeBSD is pretty much okay; but > objections may arise. > > [1] http://mail.python.org/pipermail/python-dev/2002-March/021543.html > > -- > Sincerely, > max ulidtko > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] os.ioprio_get() and os.ioprio_set()
I've recently implemented this functionality in psutil: http://code.google.com/p/psutil/issues/detail?id=147 If desired, I can contribute a patch for the os module, altough being such functions Linux-only, I'm not sure os module is the right place for them to land. Also, I've been thinking about this for quite a bit: would it be the case to add system-specific modules such as "linux" (and maybe also a "win32") to the standard library? --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
I'm sorry, I'm going to revert those checkins. They are very minor changes which I'm sure don't break anything, but I understand your complain. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/2/11 Nick Coghlan : > On Fri, Feb 11, 2011 at 11:04 PM, giampaolo.rodola > wrote: >> Author: giampaolo.rodola >> Date: Fri Feb 11 14:04:18 2011 >> New Revision: 88395 >> >> Log: >> asyncore: introduce a new 'closed' attribute to make sure that dispatcher >> gets closed only once. >> In different occasions close() might be called more than once, causing >> problems with already disconnected sockets/dispatchers. > > Giampaolo, > > This checkin and the previous one are not appropriate for the release > candidate phase of the 3.2 release. At the very least, they need to > identify the second core dev that reviewed them, as well as a > reference to the tracker issue where the RM approved them for > inclusion. > > Regards, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
Yeah, the original API design (which is very inflexible) and the lack of maintenance for many years is at the base of asyncore problems. I still think it worths some love as a stdlib module, though. For 3.3 I have in mind to revamp asyncore/asynchat a bit by introducing SSL support and finally add a scheduler (issue 1641). --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/2/11 Daniel Stutzbach : > On Fri, Feb 11, 2011 at 8:06 AM, Guido van Rossum wrote: >> >> And finally remember that asyncore is the most monkey-patched module >> in the world. :-) > > I propose that in Python 3.3 we rename asyncore to barrel_of_monkeys. > -- > Daniel Stutzbach > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
2011/2/13 Antoine Pitrou : > >> It would then be subject to python-dev development policy rather than >> twisted dev policy (which is even stricter!). Would the twisted devs >> *really* want that? We could use the same processes we have for >> "externally maintained" libraries, but they have without fail caused us >> problems. > > Oh, I agree with you. -1 on any new externally maintained library. > >> The other issue is that just because we provide an alternative doesn't >> mean that everyone automatically stops using asyncore and migrates. > > Of course. asyncore's problem is not that its a maintenance burden, it's > that it's really subpar compared to everything else out there. > That said, Giampaolo has committed to taking it forward, so perhaps the > 3.3 version of asyncore will be much (?) better. I must say that asyncore can surely be improved but not so that it can reach the flexibility offered by Twisted. Or at least, not without changing some aspects of the current API and break backward compatibility. I'll try to summarize what I think is wrong with asyncore so that maybe someone can chime in and propose ideas. Guido was right when he stated that providing a future-proof and maximally flexible design of such an API is not easy, and this is exactly the problem with asyncore. It provides a subclass-based API which doesn't work well in all those cases where I want to mix different functionallities as in: - I want a base TCP dispatcher - ...with buffered output capabilities a-la asynchat - ...which is able to limit the speed for incoming data - ...and that can also switch to SSL Although I don't use it, it seems that Twisted managed to do this by splitting the concepts of "transport" and "protocol" / "application" and by using zope.interface. At the current state, asyncore is not able to do this flexibly. It should at least separate transport and protocol, but again, I can't imagine how exactly and it would surely have a cost in terms of backward compatibility. Another problem is that asyncore is pretty low-level, and this is why the outcome is a code which looks monkey patched. Where Twisted provides a dataReceived() method, asyncore provides handle_read(): the user is supposed to override handle_read() and manually call recv() which might either fail (e.g. "retry later" or "disconnected") or succeed. The same applies for all other aspects of a TCP connection: handle_accept() -> accept(), handle_connect() -> connect() and handle_write -> send(). They all might fail and all need to be handled with care individually (see for example: http://bugs.python.org/issue6706 ). This aspect might be mitigated by providing a serie of higher lever classes (e.g. TCPServer, UDPServer, TCPConnection, UDPConnection, SSLTCPConnection) providing an API similar to: http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html ...but the need of a separation between transport and protocol layers is still needed. Last but not least, the asyncore "reactor" (asyncore.loop()) is not tied with the dispatcher. >From the dispatcher we have no reference to the reactor, hence we cannot register/unregister file descriptors, forcing the main loop to iterate over all dispatcher instances and making impossible to benefit of epoll() and kqueue(), which are crucial for scalable servers handling thousands simultaneous requests: http://bugs.python.org/issue6692 As for what we can *currently* do without going into too much trouble I can mention: http://bugs.python.org/issue10084 http://bugs.python.org/issue1641 As for Twisted core, I think it would be a nice addition for the stdlib, but for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and "taste" of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be "adapted" a bit. The main reason why I decided to use asyncore is that, despite it's huge gaps and lack of base functionnalities, I can read its source code, understand what's going on and extend it via monkey patching. It might seem a poor approach but it worked for me and couldn't do the same when I tried to use Twisted. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88505 - python/branches/py3k/Lib/ftplib.py
I'll do. 2011/2/23 Victor Stinner : > You should maybe backport this fix to Python 3.2. > > Le mardi 22 février 2011 à 20:24 +0100, giampaolo.rodola a écrit : >> Author: giampaolo.rodola >> Date: Tue Feb 22 20:24:33 2011 >> New Revision: 88505 >> >> Log: >> In FTP.close() method, make sure to also close the socket object, not only >> the file. >> >> Modified: >> python/branches/py3k/Lib/ftplib.py >> >> Modified: python/branches/py3k/Lib/ftplib.py >> == >> --- python/branches/py3k/Lib/ftplib.py (original) >> +++ python/branches/py3k/Lib/ftplib.py Tue Feb 22 20:24:33 2011 >> @@ -589,11 +589,11 @@ >> >> def close(self): >> '''Close the connection without assuming anything about it.''' >> - if self.file: >> + if self.file is not None: >> self.file.close() >> + if self.sock is not None: >> self.sock.close() >> - self.file = self.sock = None >> - >> + self.file = self.sock = None >> > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Link to issue tracker
+1, I often use that link as well. 2011/2/23 Antoine Pitrou : > > Hello, > > I think it was a slight mistake to remove the link to the issue tracker > from the sidebar in the "core development" section. Dave Beazley just > complained about it > (http://twitter.com/dabeaz/status/40397577916661760) and I think it > will probably confuse other people too. Could we add it back ? > > cheers > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r88501 - python/branches/py3k/Lib/smtplib.py
Mmmm probably. smtplib patches aren't too big/many though. Should I revert the change? 2011/2/23 Georg Brandl : > You're sure this will not cause tedious conflicts with backports? > > Georg > > On 22.02.2011 16:56, giampaolo.rodola wrote: >> Author: giampaolo.rodola >> Date: Tue Feb 22 16:56:20 2011 >> New Revision: 88501 >> >> Log: >> smtlib.py PEP8 normalization via pep8.py script. >> >> Modified: >> python/branches/py3k/Lib/smtplib.py >> >> Modified: python/branches/py3k/Lib/smtplib.py >> == >> --- python/branches/py3k/Lib/smtplib.py (original) >> +++ python/branches/py3k/Lib/smtplib.py Tue Feb 22 16:56:20 2011 >> @@ -52,15 +52,15 @@ > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r88501 - python/branches/py3k/Lib/smtplib.py
Done for Python 3.1 and 2.7. 2011/2/24 Georg Brandl : > On 24.02.2011 20:51, Giampaolo Rodolà wrote: >> Mmmm probably. smtplib patches aren't too big/many though. >> Should I revert the change? > > It's probably fine if you do the same change to the maintenance > branches as well. > > Georg > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88729 - python/branches/py3k/Modules/posixmodule.c
Thanks. I'll try to remember ACKS and NEWS in the future. =) Fixed in r88744 and r88745. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/3/4 Nick Coghlan : > On Fri, Mar 4, 2011 at 10:33 PM, Nick Coghlan wrote: >> On Fri, Mar 4, 2011 at 2:10 AM, giampaolo.rodola >> wrote: >>> Author: giampaolo.rodola >>> Date: Thu Mar 3 17:10:51 2011 >>> New Revision: 88729 >>> >>> Log: >>> Issue 11351 - apply patch by Steffen Daode Nurpmeso which should fix >>> TestSendfile.test_headers failure on OSX. >>> >>> Modified: >>> python/branches/py3k/Modules/posixmodule.c >> >> NEWS entry and a new name in ACKS? >> >> (the query regarding the lack of a NEWS entry applies to your other >> recent commits as well). > > Oops, one of them did have an entry, and a second was just a tweak to > the first one. So the NEWS query only applies to this one and the NNTP > change. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] public visibility of python-dev decisions "before it's too late" (was: PyCObject_AsVoidPtr removed from python 3.2 - is this documented?)
> Actually, why not put up a web page of "upcoming changes" somewhere, that > lists major decisions with user impact that were taken on python-dev? I think "what's new" serves this purpose properly. Usually, every time I commit a new feature, I update the what's new file as well. In fact we already have a partial "roadmap" for Python 3.3: http://docs.python.org/dev/whatsnew/3.3.html I'm not sure who else is doing the same though. If we agree that every time a new feature is added we also update the what's new file we can have such a roadmap with relatively no effort. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/3/9 Stefan Behnel : > "Martin v. Löwis", 08.03.2011 23:47: I think everything here is as it should be. People who really cared about forwards compatibility could have known, but factually, most people don't care enough. Those then learn for the first time that some feature was deprecated after it is actually removed. They then ask why it is removed, and somebody will tell them. >>> >>> I was not aware I could turn on deprecation warning for use of the C >>> API. How can I do that? >> >> Not sure that you can. When I said "could have known", I meant "by >> reading the documentation". > > I can confirm that the Cython project was as surprised of the PyCapsule > change in Python 3.2 as (I guess) most other users, and I would claim that > we are a project with one of the highest probabilities of being impacted by > C-API changes. > > Maybe the "what's new" document could at least include a link to the > relevant python-dev discussion/decision, so that fewer people have to ask > back? > > Actually, why not put up a web page of "upcoming changes" somewhere, that > lists major decisions with user impact that were taken on python-dev? > Including a link to the relevant discussion and decision. Often enough, > decisions are taken inside of huge mailing list threads that get off-topic > before someone has "the right idea" and everyone who's still there to listen > agrees. Even for people lurking around on python-dev, it's easy enough to > miss these moments. > > A publicly visible list of those decisions would > > a) make it easier for non-core developers to follow important changes on > python-dev > > b) allow potentially impacted people to bring up their POV more quickly, > e.g. during the alpha cycle of a deprecation release rather than the > following release, as in this case > > c) document the decision taking process by listing the relevant mailing list > threads > > d) help in writing the "what's new" documents > > Stefan > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python and super
:-) 2011/4/14 Antoine Pitrou > On Thu, 14 Apr 2011 08:15:10 -0500 > Benjamin Peterson wrote: > > 2011/4/14 Ricardo Kirkner : > > > Hi all, > > > > > > I recently stumbled upon an issue with a class in the mro chain not > > > calling super, therefore breaking the chain (ie, further base classes > > > along the chain didn't get called). > > > I understand it is currently a requirement that all classes that are > > > part of the mro chain behave and always call super. My question is, > > > shouldn't/wouldn't it be better, > > > if python took ownership of that part, and ensured all classes get > > > called, even if some class misbehaved? > > > > > > For example, if using a stack-like structure, pushing super calls and > > > popping until the stack was empty, couldn't this restriction be > > > removed? > > > > No. See line 2 of the Zen of Python. > > You could have quoted it explicitly :) > FWIW, line 2 is: >Explicit is better than implicit. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: os.sendfile(): on Linux if offset parameter is passed as NULL we were
No we haven't. I plan to make a unique commit for offset=None on Linux and a serie of other tests I have implemented for py-sendfile module [1]. In details test for small file, empty file and (most important) large file: http://code.google.com/p/py-sendfile/source/browse/trunk/test/test_sendfile.py?spec=svn68&r=68#296 [1] http://code.google.com/p/py-sendfile --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil 2011/4/19 Antoine Pitrou : > On Tue, 19 Apr 2011 09:47:21 +0200 > giampaolo.rodola wrote: > >> http://hg.python.org/cpython/rev/8c49f7fbba1d >> changeset: 69437:8c49f7fbba1d >> user: Giampaolo Rodola' >> date: Tue Apr 19 09:47:16 2011 +0200 >> summary: >> os.sendfile(): on Linux if offset parameter is passed as NULL we were >> erroneously returning a (bytes_sent, None) tuple instead of bytes_sent > > Do we have tests for this? > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Socket servers in the test suite
2011/4/27 Vinay Sajip : > I've been recently trying to improve the test coverage for the logging > package, > and have got to a not unreasonable point: > > logging/__init__.py 99% (96%) > logging/config.py 89% (85%) > logging/handlers.py 60% (54%) > > where the figures in parentheses include branch coverage measurements. > > I'm at the point where to appreciably increase coverage, I'd need to write > some > test servers to exercise client code in SocketHandler, DatagramHandler and > HTTPHandler. > > I notice there are no utility classes in test.support to help with this kind > of > thing - would there be any mileage in adding such things? Of course I could > add > test server code just to test_logging (which already contains some socket > server > code to exercise the configuration functionality), but rolling a test server > involves boilerplate such as using a custom RequestHandler-derived class for > each application. I had in mind a more streamlined approach where you can just > pass a single callable to a server to handle requests, e.g. as outlined in > > https://gist.github.com/945157 > > I'd be grateful for any comments about adding such functionality to e.g. > test.support. > > Regards, > > Vinay Sajip > I agree having a standard server framework for tests woul be useful, because it's something which appears quite often, (e.g. when writing functional tests). See for example: http://hg.python.org/cpython/file/b452559eee71/Lib/test/test_os.py#l1316 http://hg.python.org/cpython/file/b452559eee71/Lib/test/test_ftplib.py#l211 http://hg.python.org/cpython/file/b452559eee71/Lib/test/test_ssl.py#l844 http://hg.python.org/cpython/file/b452559eee71/Lib/test/test_smtpd.py http://hg.python.org/cpython/file/b452559eee71/Lib/test/test_poplib.py#l115 Regards --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue Tracker
2011/4/30 anatoly techtonik : > On Tue, Mar 29, 2011 at 4:37 AM, R. David Murray > wrote: >> >> The hardest part is debugging the TAL when you make a mistake, but >> even that isn't a whole lot worse than any other templating language. > > How much in % is it worse than Django templating language? > -- > anatoly t. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > Knowing both of them I can say ZPT is one of the few things I like about Zope and I find it a lot more powerful than Django templating system. Other than that, I don't see how changing the templating language can make any difference. If one does not contribute something because of the language used in templates... well, I think it wouldn't have been a particular good contribution anyway. =) --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Bring new features to older python versions
Hello everybody, at work we're using different versions of python, from 2.4 to 2.7. Because of the differences between the various versions in terms of features we have a "util.pycompat" module which basically is a copy & paste of different features which were added to stdlib in every new major version throughout the years. What we do is basically this. Instead of: from collections import namedtuple, OrderedDict import fractions ...we do: from util.pycompat.collections import namedtuple, OrderedDict from util.pycompat import fractions # py 2.6 from util.pycompat.builtins import all, any # py 2.5 # etc... This let us use different stdlib features which appeared in latest Python versions (including 3.2) throughout all our code base. Now, what I have in mind is to release this as a public module so that everyone who cannot upgrade to a recent python version can benefit of newer features. By taking a quick look at the various "what's new" documents this is a brief list of what this module would include: functools (2.5) any, all builtins (2.5) collections.defaultdict (2.5) property setters/deleters (2.6) abc (2.6) fractions (2.6) collections.OrderedDict (2.7) collections.Counter (2.7) unittest2 (2.7) functools.lru_cache (3.2) functools.total_ordering (3.2) itertools.accumulate (3.2) reprlib (3.2) contextlib.ContextDecorator (3.2) I have a couple of doubts about this though. The first one is about licensing. What I would be doing is basically copy & paste pieces of the python stdlib modules (including tests) and, where needed, adjust them so that they work with older python versions. Would this represent problem? My second doubt is about morality. Although this might be useful to those people who are forced to use older python versions, on the other hand it might represent an incentive for not upgrading (and there will be python 3.X features as well). Or maybe it won't, I don't know, point is I feel kind of guilty. =) I'd like to hear your opinions, especially about the second point. Best regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bring new features to older python versions
Thanks everybody for your feedback. I created a gcode project here: http://code.google.com/p/pycompat/ 2011/10/8 Antoine Pitrou : > There's also some stuff there that is coded in C, or that will rely on > some functionality of the core interpreter that is not easily > emulated on previous versions. But I suppose you'll find that out by > yourself. Yep, I'm still not sure what to do about this. I guess I'll just ignore that stuff in all those cases where rewriting it in python is too much effort. Toshio Kuratomi wrote: > I have a need to support a small amount of code as far back as python-2.3 > I don't suppose you're interested in that as well? ;-) I'm still not sure; 2.3 version is way too old (it doesn't even have decorators). It might make sense only in case the lib gets widely used, which I doubt. Personally, at some point I deliberately dropped support for 2.3 from all of my code/lib, mainly because I couldn't use decorators. so I don't have a real need to do this. 2011/10/9 Éric Araujo : > The issues I foresee with your lib are more technical: First, it looks > like a big bag of backported modules, classes and functions without > defined criterion for inclusion (“cool new stuff”?). I'd say the criterion for inclusion is putting in everything which can be (re)written in python 2.4, such as any, all, collections.defaultdict and property setters/deleters (2.6). Pretty much all the stuff written in C would be left out, maybe with the exception of functools module which is important (for me at least), in which case I might try to rewrite it in pure Python. I'm sharing your same doubts though. Maybe this isn't worth the effort in the first place. I'll try to write some more code and see whether this is a good candidate for a "public module". If not I'll just get back to use it as an internal "private" module. 2011/10/9 Éric Araujo : > keep on lumping new things until Python 3.4? 3.8? Won’t that become > unmanageable (boring/huge/hard)? I don't think it makes sense to go over than 3.2 version. Folks which are forced to use python 2.4 are already avoing to use 2.6 and 2.7 features, let alone 3.X only features. Plus, python 3.2 was already the latest 3.X version which still had something in common with 2.7. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3151 accepted
2011/10/12 Antoine Pitrou : > On Tue, 11 Oct 2011 18:22:43 -0400 > Barry Warsaw wrote: >> As the BDFOP for PEP 3151, I hereby accept it for inclusion into Python 3.3. >> >> Congratulations to Antoine for producing a great PEP that has broad >> acceptance >> in the Python development community, with buy-in from all the major >> implementations of Python. Antoine's branch is ready to go and it should now >> be merged into the default branch. >> >> PEP 3151 will bring some much needed sanity to this part of the standard >> exception hierarchy, and I for one look forward to being able to write code >> directly using it, one day finally eliminating most of my `import errno`s! > > Thanks Barry! > I expect to merge the PEP 3151 into default soon (it's basically ready). > > cheers > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > Thank you for having worked on this, it was a pretty huge amount of work. We'll probabily have to wait a long time before seeing libs/apps freely depending on this change without caring about backward compatibility constraints, but with this Python is a better language now. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPy 1.7 - widening the sweet spot
2011/11/21 Terry Reedy : > I strongly recommend that where it makes a difference, the pypy python3 > project target 3.3. In particular, don't reproduce the buggy narrow-build > behavior of 3.2 and before (perhaps pypy avoids this already). Do include > the new unicode capi in cpyext. I anticipate that 3.3 will see more > production use than 3.2 Is there a reason in particular? --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: fix wrong credit and issue id given in previous commit
Sorry, thanks (fixed). --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/11/22 Antoine Pitrou : > On Tue, 22 Nov 2011 13:38:03 +0100 > giampaolo.rodola wrote: >> diff --git a/Misc/ACKS b/Misc/ACKS >> --- a/Misc/ACKS >> +++ b/Misc/ACKS >> @@ -11,7 +11,7 @@ >> PS: In the standard Python distribution, this file is encoded in UTF-8 >> and the list is in rough alphabetical order by last names. >> >> -Matt Mulsow >> +Chris Clark >> Rajiv Abraham > > "The list is in rough alphabetical order by last names". > > Regarfs > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: sort last committed name in alphabetical order
Nope, the commit involving sched was the previous one. This one was just an unrelated fix. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/11/22 Nadeem Vawda : > Did you mean to also modify sched.py in this changeset? > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: sort last committed name in alphabetical order
You're right. I committed sched.py by accident. I'm going to revert it. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/11/22 Giampaolo Rodolà : > Nope, the commit involving sched was the previous one. > This one was just an unrelated fix. > > --- Giampaolo > http://code.google.com/p/pyftpdlib/ > http://code.google.com/p/psutil/ > > > 2011/11/22 Nadeem Vawda : >> Did you mean to also modify sched.py in this changeset? >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com >> > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: fix wrong credit and issue id given in previous commit
2011/11/22 Amaury Forgeot d'Arc : > Hi, > 2011/11/22 Giampaolo Rodolà >> >> Sorry, thanks (fixed). > > You also modified Lib/sched.py in the same commit. > Was it intended? If not, please revert it. > -- > Amaury Forgeot d'Arc You're right. I committed sched.py by accident. I'm going to revert it. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inconsistent script/console behaviour
Il 15 dicembre 2011 09:58, anatoly techtonik ha scritto: > 1. It is not a proposal, but a defect (well, you may argue, but please, > don't)> You can't copy/paste multiline scripts into system shell either, unless you append "\". It's likely that similar problems exists in a lot of other interactive shells (ruby?). And that makes sense to me, because they are supposed to be used interactively. It might be good to change this? Maybe. Is the current behavior objectively wrong? No, in my opinion. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Il 25 febbraio 2012 21:23, Armin Ronacher ha scritto: > Hi, > > I just uploaded PEP 414 which proposes am optional 'u' prefix for string > literals for Python 3. > > You can read the PEP online: http://www.python.org/dev/peps/pep-0414/ > > This is a followup to the discussion about this topic here on the > mailinglist and on twitter/IRC over the last few weeks. > > > Regards, > Armin If the main point of this proposal is avoiding an explicit 2to3 run on account of 2to3 being too slow then I'm -1. That should be fixed at 2to3 level, not at python syntax level. A common strategy to distribute code able to run on both python 2 and python 3 is using the following hack in setup.py: http://docs.python.org/dev/howto/pyporting.html#during-installation That's what I used in psutil and it works just fine. Also, I believe it's the *right* strategy as it lets you freely write python 2 code and avoid using ugly hacks such as "sys.exc_info()[1]" and "if PY3: ..." all around the place. 2to3 might be slow but introducing workarounds encouraging not to use it is only going to cause a proliferation of ugly and hackish code in the python ecosystem. Now, psutil is a relatively small project and the 2to3 conversion doesn't take much time. Having users "unawarely" run 2to3 at installation time is an acceptable burden in terms of speed. That's going to be different on larger code bases such as Twisted's. One way to fix that might be making 2to3 generate and rely on a "2to3.diff" file containing all the differences. That would be generated the first time "python setup.py build/install" is run and then partially re-calculated every time a file is modified. Third-party library vendors can include 2to3.diff as part of the tarball they distribute so that the end user won't experience any slow down deriving from the 2to3 conversion. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Il 28 febbraio 2012 13:19, Antoine Pitrou ha scritto: > > Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : >> If you're using separate branches, then your Python 2 code isn't being >> made forward compatible with Python 3. Yes, it avoids making your >> Python 2 code uglier, but it means maintaining two branches in >> parallel until you drop Python 2 support. > > IMO, maintaining two branches shouldn't be much more work than > maintaining hacks so that a single codebase works with two different > programming languages. Would that mean distributing 2 separate tarballs? How would tools such as easy_install and pip work in respect of that? Is there a naming convention they can rely on? --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 414 - Unicode Literals for Python 3
Il 28 febbraio 2012 15:20, Ezio Melotti ha scritto: > On 28/02/2012 14.19, Antoine Pitrou wrote: >> >> Le mardi 28 février 2012 à 22:14 +1000, Nick Coghlan a écrit : >>> >>> If you're using separate branches, then your Python 2 code isn't being >>> made forward compatible with Python 3. Yes, it avoids making your >>> Python 2 code uglier, but it means maintaining two branches in >>> parallel until you drop Python 2 support. >> >> IMO, maintaining two branches shouldn't be much more work than >> maintaining hacks so that a single codebase works with two different >> programming languages. > > > +10 > > For every CPython bug that I fix I first apply the patch on 2.7, then on 3.2 > and then on 3.3. > Most of the time I don't even need to change anything while applying the > patch to 3.2, sometimes I have to do some trivial fixes. This is also true > for another personal 12kloc project* where I'm using the two-branches > approach. > > For me, the costs of having two branches are: > 1) a one-time conversion when the Python3-compatible branch is created (can > be done easily with 2to3); > 2) merging the fix I apply to the Python2 branch (and with modern DVCS this > is not really an issue). > > With the shared code base approach, the costs are: > 1) a one-time conversion to "fix" the code base and make it run on both 2.x > and 3.x; > 2) keep using and having to deal with hacks in order to keep it running. > > With the first approach, you also have two clean and separate code bases, > with no hacks; when you stop using Python 2, you end up with a clean Python > 3 branch. > The one-time conversion also seems easier in the first case. > > (Note: there are also other costs -- e.g. releasing -- that I haven't > considered because they don't affect me personally, but I'm not sure they > are big enough to make the two-branches approach worse.) They are. With that kind of approach you're basically forced to include the python version number as part of the tarball name (e.g. foo-0.3.1-py2.tar.gz and foo-0.3.1-py3.tar.gz). Just to name one, that means "foo" can't be installed via pip/easy_install. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a frozendict builtin type
Il 01 marzo 2012 02:45, Raymond Hettinger ha scritto: > > On Feb 29, 2012, at 4:23 PM, Victor Stinner wrote: > > One of my colleagues implemented recently its own frozendict class > (which the "frozendict" name ;-) > > > I write new collection classes all the time. > That doesn't mean they warrant inclusion in the library or builtins. > There is a use case for ListenableSets and ListenableDicts -- do we > need them in the library? I think not. How about case insensitive > variants? > I think not. There are tons of recipes on ASPN and on PyPI. > That doesn't make them worth adding in to the core group of types. > > As core developers, we need to place some value on language > compactness and learnability. The language has already gotten > unnecessarily fat -- it is the rare Python programmer who knows > set operations on dict views, new-style formatting, abstract base classes, > contextlib/functools/itertools, how the with-statement works, > how super() works, what properties/staticmethods/classmethods are for, > differences between new and old-style classes, Exception versus > BaseException, > weakreferences, __slots__, chained exceptions, etc. > > If we were to add another collections type, it would need to be something > that powerfully adds to the expressivity of the language. Minor variants > on what we already have just makes that language harder to learn and > remember > but not providing much of a payoff in return. > > > Raymond > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > +1 --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASED] Python 3.3.0 alpha 1
Il 06 marzo 2012 20:43, Jim J. Jewett ha scritto: > > > In http://mail.python.org/pipermail/python-dev/2012-March/117348.html > Georg Brandl posted: > >> Python 3.3 includes a range of improvements of the 3.x series, as well as >> easier >> porting between 2.x and 3.x. Major new features in the 3.3 release series >> are: > > As much as it is nice to just celebrate improvements, I think > readers (particularly on the download page > http://www.python.org/download/releases/3.3.0/ ) would be better > served if there were an additional point about porting and the > hash changes. > > http://docs.python.org/dev/whatsnew/3.3.html#porting-to-python-3-3 > also failed to mention this, and even the changelog didn't seem to > warn people about failing tests or tell them how to work around it. > > Perhaps something like: > > Hash Randomization (issue 13703) is now on by default. Unfortunately, > this does break some tests; it can be temporarily turned off by setting > the environment variable PYTHONHASHSEED to "0" before launching python. > > > -jJ > > -- > > If there are still threading problems with my replies, please > email me with details, so that I can try to resolve them. -jJ That's why I once proposed to include whatsnew.rst changes every time a new feature is added/committed. Assigning that effort to the release manager or whoever is supposed to take care of this, is both impractical and prone to forgetfulness. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] os.path.exists() / os.path.isdir() inconsistency when dealing with gvfs directories
I've just noticed a strange behavior when dealing with gvfs filesystems: giampaolo@ubuntu:~$ python -c "import os; print(os.path.exists('/home/giampaolo/.gvfs'))" True giampaolo@ubuntu:~$ sudo su root@ubuntu:~# python -c "import os; print(os.path.exists('/home/giampaolo/.gvfs'))" False This is due to os.stat() which internally fails with PermissionError (EACCES). The same problem exists with os.path.isdir() which will return True as limited user and False as root. I'm not sure what's best to do here nor I know if there are other cases other than when dealing with gvfs which can produce similar behaviors but here's an idea: - make os.path.exists() return True in case of PermissionError because that's supposed to mean there's an existing path to deny access to - fix isdir(), islink(), isfile() documentation pointing out that in case of EACCES/EPERM or when dealing with exotic paths/fs it may return incorrect results. Comments? --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.exists() / os.path.isdir() inconsistency when dealing with gvfs directories
2012/6/27 Cameron Simpson : > On 27Jun2012 01:49, Giampaolo Rodolà wrote: > | I've just noticed a strange behavior when dealing with gvfs filesystems: > | > | giampaolo@ubuntu:~$ python -c "import os; > | print(os.path.exists('/home/giampaolo/.gvfs'))" > | True > | giampaolo@ubuntu:~$ sudo su > | root@ubuntu:~# python -c "import os; > | print(os.path.exists('/home/giampaolo/.gvfs'))" > | False > | > | This is due to os.stat() which internally fails with PermissionError > (EACCES). > | The same problem exists with os.path.isdir() which will return True as > | limited user and False as root. > | I'm not sure what's best to do here nor I know if there are other > | cases other than when dealing with gvfs which can produce similar > | behaviors but here's an idea: > | > | - make os.path.exists() return True in case of PermissionError because > | that's supposed to mean there's an existing path to deny access to > > Definitely not. > > Firstly, if I ask about "a/b/c" and am denied access to "a/b", then it > would be a lie to say "c" exists - it may not. Right, I wasn't taking that into account. > So I'd be +0.5 for making the docs more clear that True is reliable and > False may merely mean "could not access". +1. I was about to propose a 'strict' parameter which lets the exception propagate in case of errno != EACCES/EPERM but a doc fix is probably just fine. I'll file a bug report later today. Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] os.path.exists() / os.path.isdir() inconsistency when dealing with gvfs directories
2012/6/27 Nick Coghlan : > If someone wants to see the error details, they should use os.stat directly > rather than an existence check. This is now tracked at http://bugs.python.org/issue15221 Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Snakebite build slaves and developer SSH/GPG public keys
> For committers on other Python projects like Buildbot, Django and > Twisted that may be reading this -- yes, the plan is to give you > guys Snakebite access/slaves down the track too. I'll start looking > into that after I've finished setting up the remaining slaves for > Python. (Setting up a keys repo will definitely help (doesn't have > to be hg -- feel free to use svn/git/whatever, just try and follow > the same layout).) This is so great! I've been looking forward to this for a long time and kept visiting the site every now and then to see if there was any progress. I'd surely use this for psutil if you'll let me. Also, at some point I would suggest to introduce the possibility to donate some money in order to help supporting what I think must be a pretty complex infrastructure requiring a lot of resources, both in terms of hardware and time/labor. Regards --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ 2012/8/23 Trent Nelson : > Hi folks, > > I've set up a bunch of Snakebite build slaves over the past week. > One of the original goals was to provide Python committers with > full access to the slaves, which I'm still keen on providing. > > What's a nice simple way to achieve that in the interim? Here's > what I was thinking: > > - Create a new hg repo: hg.python.org/keys. > > - Committers can push to it just like any other repo (i.e. > same ssh/authz configuration as cpython). > > - Repo is laid out as follows: > keys/ > / > ssh (ssh public key) > gpg (gpg public key) > > - Prime the repo with the current .ssh/authorized_keys > (presuming you still use the --tunnel-user facility?). > > That'll provide me with everything I need to set up the relevant > .ssh/authorized_keys stuff on the Snakebite side. GPG keys will > be handy if I ever need to send passwords over e-mail (which I'll > probably have to do initially for those that want to RDP into the > Windows slaves). > > Thoughts? > > As for the slaves, here's what's up and running now: > > - AMD64 Mountain Lion [SB] > - AMD64 FreeBSD 8.2 [SB] > - AMD64 FreeBSD 9.1 [SB] > - AMD64 NetBSD 5.1.2 [SB] > - AMD64 OpenBSD 5.1 [SB] > - AMD64 DragonFlyBSD 3.0.2 [SB] > - AMD64 Windows Server 2008 R2 SP1 [SB] > - x86 NetBSD 5.1.2 [SB] > - x86 OpenBSD 5.1 [SB] > - x86 DragonFlyBSD 3.0.2 [SB] > - x86 Windows Server 2003 R2 SP2 [SB] > - x86 Windows Server 2008 R2 SP1 [SB] > > All the FreeBSD ones use ZFS, all the DragonFly ones use HAMMER. > DragonFly, NetBSD and OpenBSD are currently reporting all sorts > of weird and wonderful errors, which is partly why I want to set > up ssh access sooner rather than later. > > Other slaves on the horizon (i.e. hardware is up, OS is installed): > > - Windows 8 x64 (w/ VS2010 and VS2012) > - HP-UX 11iv2 PA-RISC > - HP-UX 11iv3 Itanium (64GB RAM) > - AIX 5.3 RS/6000 > - AIX 6.1 RS/6000 > - AIX 7.1 RS/6000 > - Solaris 9 SPARC > - Solaris 10 SPARC > > Nostalgia slaves that probably won't ever see green: > - IRIX 6.5.33 MIPS > - Tru64 5.1B Alpha > > If anyone wants ssh access now to the UNIX platforms in order to > debug/test, feel free to e-mail me directly with your ssh public > keys. > > For committers on other Python projects like Buildbot, Django and > Twisted that may be reading this -- yes, the plan is to give you > guys Snakebite access/slaves down the track too. I'll start looking > into that after I've finished setting up the remaining slaves for > Python. (Setting up a keys repo will definitely help (doesn't have > to be hg -- feel free to use svn/git/whatever, just try and follow > the same layout).) > > Regards, > > Trent "that-took-a-bit-longer-than-expected" Nelson. > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed schedule for Python 3.4
2012/10/3 Larry Hastings : > Other proposed large-scale changes: > [...] > * A standard event-loop interface (PEP by Jim Fulton pending) Really? Was this discussed somewhere? I'd like to know more about it. --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com