Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
On 07/06/2014 02:13, Donald Stufft wrote: On Jun 6, 2014, at 9:05 PM, Terry Reedy wrote: On 6/6/2014 6:47 PM, Nathaniel Smith wrote: On Fri, Jun 6, 2014 at 11:43 PM, Sturla Molden wrote: Brett Cannon wrote: Nope. A new minor release of Python is a massive undertaking which is why we have saved ourselves the hassle of doing a Python 2.8 or not giving a clear signal as to when Python 2.x will end as a language. Why not just define Python 2.8 as Python 2.7 except with a newer compiler? I cannot see why that would be massive undertaking, if changing compiler for 2.7 is neccesary anyway. This would require recompiling all packages on OS X and Linux, even though nothing had changed. If you are suggesting that a Windows compiler change should be invisible to non-Windows users, I agree. Let us assume that /pcbuild remains for those who have vc2008 and that /pcbuild14 is added (and everything else remains as is). Then the only other thing that would change is the Windows installer released on Python.org. Call than 2.7.9W or whatever on the download site and interactive startup message to signal that something is different. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io How are packaging tools supposed to cope with this? AFAIK there is nothing in most of them to deal with a X.Y.Z release suddenly dealing with a different compiler. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA Potentially completely stupid suggestion to get people thinking (or die laughing :) , but would it be possible to use hex digits, such that 2.7.A was the first release on Windows with the different compiler? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
On Sat, Jun 7, 2014 at 7:05 AM, Donald Stufft wrote: > > I don’t particularly care too much though, I just think that bumping > the compiler in a 2.7.Z release is a really bad idea and that either > of the other two options are massively better. +1 -- Giampaolo - http://grodola.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-ideas] Expose `itertools.count.start` and implement `itertools.count.__eq__` based on it, like `range`.
On 7 June 2014 19:36, Ram Rachum wrote: > My need is to have an infinite immutable sequence. I did this for myself by > creating a simple `count`-like stateless class, but it would be nice if that > behavior was part of `range`. Handling esoteric use cases like it sounds yours was is *why* user defined classes exist. It does not follow that "I had to write a custom class to solve my problem" should lead to a standard library or builtin changing unless you can make a compelling case for: * the change being a solution to a common problem that a lot of other people also have. "I think it might be nice" and "it would have been useful to me to help solve this weird problem I had that one time" isn't enough. * the change fitting in *conceptually* with the existing language and tools. In this case, "infinite sequence" is a fundamentally incoherent concept in Python - len() certainly won't work, and negative indexing behaviour is hence not defined. By contrast, since iterables and iterators aren't required to support len() the way sequences are, infinite iterable and infinite iterator are both perfectly well defined. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] namedtuple implementation grumble
Hi All, I've been trying to add support for explicit comparison of namedtuples into testfixtures and hit a problem which lead me to read the source and be sad. Rather than the mixin and class assembly in the function I expected to find, I'm greeted by an exec of a string. Curious as to what lead to that implementation approach? What does it buy that couldn't have been obtained by a mixin providing the functionality? In my case, that's somewhat irrelevant, I'm looking to store a comparer in a registry that would get used for all namedtuples, but I have nothing to key that off, there are no shared bases other than object and tuple. I guess I could duck-type it based on the _fields attribute but that feels implicit and fragile. What do you guys suggest? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
On Fri, 06 Jun 2014 19:50:57 +0100, Chris Withers wrote: > I've been trying to add support for explicit comparison of namedtuples > into testfixtures and hit a problem which lead me to read the source and > be sad. > > Rather than the mixin and class assembly in the function I expected to > find, I'm greeted by an exec of a string. > > Curious as to what lead to that implementation approach? What does it > buy that couldn't have been obtained by a mixin providing the functionality? > > In my case, that's somewhat irrelevant, I'm looking to store a comparer > in a registry that would get used for all namedtuples, but I have > nothing to key that off, there are no shared bases other than object and > tuple. > > I guess I could duck-type it based on the _fields attribute but that > feels implicit and fragile. > > What do you guys suggest? I seem to remember a previous discussion that concluded that duck typing based on _fields was the way to go. (It's a public API, despite the _, due to name-tuple's attribute namespacing issues.) --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
On Fri, Jun 06, 2014 at 07:50:57PM +0100, Chris Withers wrote: > Hi All, > > I've been trying to add support for explicit comparison of namedtuples > into testfixtures and hit a problem which lead me to read the source and > be sad. > > Rather than the mixin and class assembly in the function I expected to > find, I'm greeted by an exec of a string. > > Curious as to what lead to that implementation approach? What does it > buy that couldn't have been obtained by a mixin providing the functionality? namedtuple started off as a recipe on ActiveState by Raymond Hettinger. Start here: http://code.activestate.com/recipes/500261-named-tuples/?in=user-178123 -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
On 7 June 2014 04:50, Chris Withers wrote: > Curious as to what lead to that implementation approach? What does it buy > that couldn't have been obtained by a mixin providing the functionality? In principle, you could get the equivalent of collections.namedtuple through dynamically constructed classes. In practice, that's actually easier said than done, so the fact the current implementation works fine for almost all purposes acts as a powerful disincentive to rewriting it. The current implementation is also *really* easy to understand, while writing out the dynamic type creation explicitly would likely require much deeper knowledge of the type machinery to follow. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
Le 07/06/2014 09:25, R. David Murray a écrit : On Fri, 06 Jun 2014 19:50:57 +0100, Chris Withers wrote: I've been trying to add support for explicit comparison of namedtuples into testfixtures and hit a problem which lead me to read the source and be sad. Rather than the mixin and class assembly in the function I expected to find, I'm greeted by an exec of a string. Curious as to what lead to that implementation approach? What does it buy that couldn't have been obtained by a mixin providing the functionality? In my case, that's somewhat irrelevant, I'm looking to store a comparer in a registry that would get used for all namedtuples, but I have nothing to key that off, there are no shared bases other than object and tuple. I guess I could duck-type it based on the _fields attribute but that feels implicit and fragile. What do you guys suggest? I seem to remember a previous discussion that concluded that duck typing based on _fields was the way to go. (It's a public API, despite the _, due to name-tuple's attribute namespacing issues.) There could be many third-party classes with a _fields member, so that sounds rather fragile. There doesn't seem to be any technical reason barring the addition of a common base class for namedtuples. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Division of tool labour in porting Python 2 code to 2/3
On Fri, Jun 6, 2014 at 7:37 PM, Brett Cannon wrote: > After Glyph and Alex's email about their asks for assisting in writing > Python 2/3 code, it got me thinking about where in the toolchain various > warnings and such should go in order to help direct energy to help develop > whatever future toolchain to assist in porting. > > There seems to be three places where issues are/can be caught once a project > has embarked down the road of 2/3 source compatibility: > > -3 warnings > Some linter tool Pylint could help here. We already have a couple of checks which addresses the issue of porting between Python 2 and 3, checks like: raising-string old-style-class slots-on-old-class super-on-old-class old-raise-syntax old-ne-operator lowercase-l-suffix backtick unpacking-in-except indexing-exception property-on-old-class There was an idea on Pylint's bugtracker to implement a plugin for Python 2, with warnings dedicated to porting and this solution seems easier than the alternatives. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
Once 7 Jun 2014 06:19, "Nick Coghlan" wrote: > > On 7 June 2014 15:05, Donald Stufft wrote: > > I don’t particularly care too much though, I just think that bumping > > the compiler in a 2.7.Z release is a really bad idea and that either > > of the other two options are massively better. > > It is *incredibly* unlikely that backwards compatibility with binary > extensions will be broken within the Python 2.7 series - there's a > reason we said "No" when the Stackless folks were asking about it a > while back. Instead, the toolchain availability problem is currently > being tackled by trying to make suitable build toolchains more readily > available (both the official VS 2008 toolchain and alternative open > source toolchains), and by reducing the reliance on building from > source for end users. A third piece of the puzzle could potentially be the availability of automated wheel-building services. (Personally I still haven't successfully managed to build windows wheels for my own packages, and envy my R-using colleagues whose PyPi equivalent does the building for them.) -n ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Division of tool labour in porting Python 2 code to 2/3
On Sat Jun 07 2014 at 9:11:54 AM, Claudiu Popa wrote: > On Fri, Jun 6, 2014 at 7:37 PM, Brett Cannon wrote: > > After Glyph and Alex's email about their asks for assisting in writing > > Python 2/3 code, it got me thinking about where in the toolchain various > > warnings and such should go in order to help direct energy to help > develop > > whatever future toolchain to assist in porting. > > > > There seems to be three places where issues are/can be caught once a > project > > has embarked down the road of 2/3 source compatibility: > > > > -3 warnings > > Some linter tool > > > Pylint could help here. We already have a couple of checks which > addresses the issue of porting between Python 2 and 3, checks like: > > raising-string > old-style-class > slots-on-old-class > super-on-old-class > old-raise-syntax > old-ne-operator > lowercase-l-suffix > backtick > unpacking-in-except > indexing-exception > property-on-old-class > > There was an idea on Pylint's bugtracker to implement a plugin for > Python 2, with warnings dedicated to porting and this solution seems > easier than the alternatives. > Yes, pylint is definitely an option. I have not looked at how hard it would be to write the rules, though, and how easy it would be to run with just those rules (if I remember correctly pylint can take a config, but I have not run it manually in a while). Having something which walked the 2.7 CST or AST wouldn't be difficult to write either, so it's just a matter of balance of work required. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Moving Python 3.5 on Windows to a new compiler
One more possible concern that I just thought of is the availability of the build tools on Windows Vista and Windows 7 RTM (that is, without SP1). I'd have to check, but I don't believe anything after VS 2012 is supported on Vista and it's entirely possible that installation is blocked. This may be a non-issue. VC14 still has the "XP mode" that avoids using new APIs, so compiled Python will run fine, but it may be the case that the compiler doesn't (if we manage to get a separate, compiler-only package, that is. VS itself is definitely unusable). I assume gcc/clang will continue to support earlier OSs, so hopefully by the time 3.5 is getting early releases there will be an option for building extensions. I doubt anyone on this list is stuck on Vista or in a position where they can't keep Win7 updated, but do we know of any environments where this may be a problem? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes
On Sat, Jun 7, 2014 at 1:37 AM, Greg Ewing wrote: > Julian Taylor wrote: >> >> tp_can_elide receives two objects and returns one of three values: >> * can work inplace, operation is associative >> * can work inplace but not associative >> * cannot work inplace > > > Does it really need to be that complicated? Isn't it > sufficient just to ask the object potentially being > overwritten whether it's okay to overwrite it? > I.e. a parameterless method returning a boolean. For the numpy case, we really need to see all the operands, *and* know what the operation in question is. Consider tmp1 = np.ones((3, 1)) tmp2 = np.ones((1, 3)) tmp1 + tmp2 which returns an array with shape (3, 3). Both input arrays are temporaries, but neither of them can be stolen to use for the output array. Or suppose 'a' is an array of integers and 'b' is an array of floats, then 'a + b' and 'a += b' have very different results (the former upcasts 'a' to float, the latter has to either downcast 'b' to int or raise an error). But the casting rules depend on the particular input types and the particular operation -- operations like & and << want to cast to int, < and > return bools, etc. So one really needs to know all the details of the operation before one can determine whether temporary elision is possible. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-ideas] Expose `itertools.count.start` and implement `itertools.count.__eq__` based on it, like `range`.
On Sat, Jun 7, 2014 at 5:50 AM, Nick Coghlan wrote: > On 7 June 2014 19:36, Ram Rachum wrote: > > My need is to have an infinite immutable sequence. I did this for myself > by > > creating a simple `count`-like stateless class, but it would be nice if > that > > behavior was part of `range`. > > Handling esoteric use cases like it sounds yours was is *why* user > defined classes exist. It does not follow that "I had to write a > custom class to solve my problem" should lead to a standard library or > builtin changing unless you can make a compelling case for: > > * the change being a solution to a common problem that a lot of other > people also have. "I think it might be nice" and "it would have been > useful to me to help solve this weird problem I had that one time" > isn't enough. > * the change fitting in *conceptually* with the existing language and > tools. In this case, "infinite sequence" is a fundamentally incoherent > concept in Python - len() certainly won't work, and negative indexing > behaviour is hence not defined. By contrast, since iterables and > iterators aren't required to support len() the way sequences are, > infinite iterable and infinite iterator are both perfectly well > defined. > With all due respect, “"infinite sequence" is a fundamentally incoherent concept in Python” is a bit hyperbolic. It would be perfectly reasonable to have them, but they're not defined (yet). > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
On 6/7/2014 7:50 AM, Antoine Pitrou wrote: Le 07/06/2014 09:25, R. David Murray a écrit : On Fri, 06 Jun 2014 19:50:57 +0100, Chris Withers wrote: I guess I could duck-type it based on the _fields attribute but that feels implicit and fragile. What do you guys suggest? I seem to remember a previous discussion that concluded that duck typing based on _fields was the way to go. (It's a public API, despite the _, due to name-tuple's attribute namespacing issues.) There could be many third-party classes with a _fields member, so that sounds rather fragile. There doesn't seem to be any technical reason barring the addition of a common base class for namedtuples. Regards Antoine. A common base class sounds like a good idea, to me, at a minimum, to help identify all the namedtuple derivatives. On 6/7/2014 7:46 AM, Nick Coghlan wrote: On 7 June 2014 04:50, Chris Withers wrote: Curious as to what lead to that implementation approach? What does it buy that couldn't have been obtained by a mixin providing the functionality? In principle, you could get the equivalent of collections.namedtuple through dynamically constructed classes. In practice, that's actually easier said than done, so the fact the current implementation works fine for almost all purposes acts as a powerful disincentive to rewriting it. The current implementation is also *really* easy to understand, while writing out the dynamic type creation explicitly would likely require much deeper knowledge of the type machinery to follow. I wonder if the dynamically constructed classes approach could lead to the same space and time efficiencies... seems like I recall there being a discussion of efficiency, I think primarily space efficiency, as a justification for the present implementation. namedtuple predates of the improvements in metaclasses, also, which may be a justification for the present implementation. I bumped into namedtuple when I first started coding in Python, I was looking for _some way_, _any way_ to achieve an unmutable class with named members, and came across Raymond's recipe, which others have linked to... and learned, at the time, that he was putting it into Python stdlib. I found it far from "*really* easy to understand", although at that point in my Python knowledge, I highly doubt a metaclass implementation would have been easier to understand... but learning metaclasses earlier than I did might have been good for my general understanding of Python, and more useful in the toolbox than an implementation like namedtuple. I did, however, find and suggest a fix for a bug in the namedtuple implementation that Raymond was rather surprised that he had missed, although I would have to pick through the email archives to remember now what it was, or any other details about it... but it was in time to get fixed before the first release of Python that included namedtuple, happily. I wouldn't be opposed to someone rewriting namedtuple using metaclasses, to compare the implementations from an understandability and from an efficiency standpoint... but I don't think my metaclass skills are presently sufficient to make the attempt myself. I also seem to recall that somewhere in the (lengthy) Enum discussions, that Enum uses a technique similar to namedtuple, again for an efficiency reason, even though it also uses metaclasses in its implementation. I wonder if, if the reasons were well understood by someone that understand Python internals far better than I do, if they point out some capability that is missing from metaclasses that lead to these decisions to use string parsing and manipulation as a basis for implementing classes with metaclass-like behaviors, yet not use the metaclass feature set to achieve those behaviors. Glenn ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
Hello, On Sat, 07 Jun 2014 12:42:32 -0700 Glenn Linderman wrote: > On 6/7/2014 7:50 AM, Antoine Pitrou wrote: > > Le 07/06/2014 09:25, R. David Murray a écrit : > >> On Fri, 06 Jun 2014 19:50:57 +0100, Chris Withers > >> wrote: > >>> I guess I could duck-type it based on the _fields attribute but > >>> that feels implicit and fragile. > >>> > >>> What do you guys suggest? > >> > >> I seem to remember a previous discussion that concluded that duck > >> typing based on _fields was the way to go. (It's a public API, > >> despite the _, due to name-tuple's attribute namespacing issues.) > > > > There could be many third-party classes with a _fields member, so > > that sounds rather fragile. > > There doesn't seem to be any technical reason barring the addition > > of a common base class for namedtuples. > > > > Regards > > > > Antoine. > > A common base class sounds like a good idea, to me, at a minimum, to > help identify all the namedtuple derivatives. I'm perplexed - isn't "tuple" such common base class? And checking for both "tuple" base class and "_fields" member will identify it with ~same probability as a check for special base type (because it's fair to say that if someone *both* subclassed a builtin type and add _fields member, then they wanted it to be treated as namedtuple). [] -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] cpython and python debugger documentation
Hi, I am interested in learning how the cpython interpreter is designed and implemented, and also how the python debugger works internally. My ultimate purpose is to modify them for my distributed computing needs. Are there any documentations on these please? I have done some goggling but failed to find anything useful. Thanks you very much for your help! -Le ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [numpy wishlist] Interpreter support for temporary elision in third-party classes
Nathaniel Smith wrote: For the numpy case, we really need to see all the operands, *and* know what the operation in question is... Okay, I see what you mean now. Given all that, it might be simpler just to have the method perform the operation itself if it can. It has all the information necessary to do so, after all. This would also make it possible for the inplace operators to have different semantics from temp-elided non-inplace ones if desired. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] namedtuple implementation grumble
On 8 Jun 2014 05:44, "Glenn Linderman" wrote: > > I wonder if the dynamically constructed classes approach could lead to the same space and time efficiencies... seems like I recall there being a discussion of efficiency, I think primarily space efficiency, as a justification for the present implementation. namedtuple predates of the improvements in metaclasses, also, which may be a justification for the present implementation. As far as I am aware, there's nothing magical in the classes namedtuple creates that would require a custom metaclass - it's just that what it does would likely be even harder to read if written out explicitly rather than letting the compiler & eval loop deal with it. However, we've drifted off topic for python-dev at this point. If anyone wanted to experiment with alternative implementations, python-ideas would be the place to discuss that. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
