Re: [Python-Dev] cpython: Remove some extraneous parentheses and swap the comparison order to
Terry Reedy wrote: On 6/7/2011 5:35 PM, David Malcolm wrote: I know that this style is unpopular, but if it helps, try mentally pronouncing "==" in C as "is the value of". In this example, when I read that line, my mind is thinking: "if 'u' is the value of typecode" After ~12 years of doing this, it comes naturally. I appreciate that this may come across as weird though :) Whereas I read it as 'has the value' (or just 'is' ;=). Am I the only one who reads == as "equals"? -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Can we improve support for abstract base classes with desciptors
I would like to try to address some shortfalls with the way python deals with
abstract base classes containing descriptors. I originally was just concerned
with improving support for defining abstract properties with the decorator
syntax and converting between abstract and concrete properties, but recently
realized that the problem extends to descriptors in general.
ABCs
First, a bit of background may be in order. An abstract base class is defined
by specifying its metaclass as ABCMeta (or a subclass thereof)::
class MyABC(metaclass=ABCMeta):
@abstractmethod
def foo(self):
pass
When trying to instantiate MyABC or any of its subclasses, ABCMeta inspects the
current class namespace for items tagged with __isabstractmethod__=True::
class ABCMeta(type):
#[...]
def __new__(mcls, name, bases, namespace):
cls = super().__new__(mcls, name, bases, namespace)
# Compute set of abstract method names
abstracts = {name
for name, value in namespace.items()
if getattr(value, "__isabstractmethod__", False)}
ABCMeta then checks if any of the base classes define any items tagged with
__isabstractmethod__ and whether they remain abstract in the current
class namespace::
for base in bases:
for name in getattr(base, "__abstractmethods__", set()):
value = getattr(cls, name, None)
if getattr(value, "__isabstractmethod__", False):
abstracts.add(name)
cls.__abstractmethods__ = frozenset(abstracts)
In Objects/typeobject.c, __abstractmethods__ is actually a descriptor, and
setting it gives the type a chance to set an internal flag specifying if it
has any abstract methods defined. When object_new is called in typeobject.c,
the flag is checked and an error is raised if any abstract methods were
identified.
Issues with ABCs and descriptors
In order for this scheme to work, ABCMeta needs to identify all of the abstract
methods, but there are some limitations when we consider descriptors. For
example, Python's property is a composite object, whose behavior is defined by
the getter, setter, and deleter methods with which it is composed. Since there
is already an @abstractmethod decorator, I would have suspected that defining
abstract properties would be intuitive::
class MyABC(metaclass=ABCMeta):
@abstractmethod
def _get_foo(self):
pass
@abstractmethod
def _set_foo(self, val):
pass
foo = property(_get_foo, _set_foo)
@property
@abstractmethod
def bar(self):
pass
@bar.setter
@abstractmethod
def bar(self, val):
pass
Ideally, one would want the flexibility of defining a concrete getter and an
abstract setter, for example. However, ABCMeta does not inspect the descriptors
of a class to see if they contain any abstract methods. It only inspects the
descriptor itself for a True __isabstractmethod__ attribute. This places the
burdon on every descriptor implementation to provide its own support for ABC
compatibility. For example, support for abstract properties was attempted by
adding abstractproperty to the abc module. abstractproperty subclasses the
property builtin (as opposed to the relationship between every other abstract
and concrete class in the python language). Here is the definition of
abstractproperty, in its entirety (modulo docstrings)::
class abstractproperty(property):
__isabstractmethod__ = True
A number of problems manifest with this approach, and I think they all can be
traced to the fact that the abstractedness of a descriptor is currently not
dependent upon the abstractedness of the methods with which it is
composed. The documentation for abstractproperty doesn't suggest using
@abstractmethod::
class C(metaclass=ABCMeta):
def getx(self): ...
def setx(self, value): ...
x = abstractproperty(getx, setx)
which leads to Issue #1: What is abstract about C.x? How does a subclass of C
know whether it needs to override the getter or setter?
Issue #2: The decorator syntax cannot be used to convert an abstract property
into a concrete one. (This relates to Issue #1: how would a descriptor even know
when such a conversion would be appropriate?) Running the following code::
from abc import ABCMeta, abstractmethod, abstractproperty
class AbstractFoo(metaclass=ABCMeta):
@abstractproperty
def bar(self):
return 1
@bar.setter
def bar(self, val):
pass
class ConcreteFoo(AbstractFoo):
@AbstractFoo.bar.getter
def bar(self):
return 1
@bar.setter
def bar(self, val):
pass
foo = ConcreteFoo()
yields::
TypeError: Can't instantiate abstrac
Re: [Python-Dev] The socket HOWTO
On Wed, Jun 8, 2011 at 3:37 AM, Eli Bendersky wrote: > Just be careful not to reproduce http://www.apress.com/9781590593714 :-) > These things tend to get out of hand very quickly. At the level Glyph and Martin are talking about, you're more likely to end up with http://authors.phptr.com/tanenbaumcn4/ :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The socket HOWTO
On Tue, Jun 7, 2011 at 1:54 PM, Glyph Lefkowitz wrote: > how is UDP different from TCP? > > The phrase "UDP" never appears in the HOWTO. DGRAM sockets get a brief > mention as "anything else" in the sentence: "... you’ll get better behavior > and performance from a STREAM socket than anything else ...". (To be fair, > I do endorse teaching that "the difference between TCP and UDP is that you > should not use UDP" to anyone not sufficiently advanced to read the relevant > reference documentation themselves.) And if UDP starts sounding tempting due to excessively high latency, these days it's worth looking up the specs for the interplanetary internet instead. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Remove some extraneous parentheses and swap the comparison order to
On Wed, Jun 8, 2011 at 7:35 AM, David Malcolm wrote: > After ~12 years of doing this, it comes naturally. I appreciate that > this may come across as weird though :) I actually thought Brett's rationale in the checkin comment was reasonable (if you get in the habit of putting constants on the left, then the classic "'=' instead of '=='" typo is a compiler error instead of a reassignment). Call it a +0 in favour of letting people put constants on the left in C code if they prefer it that way, so long as any given if/elif chain is consistent in the style it uses. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Remove some extraneous parentheses and swap the comparison order to
On Wed, Jun 8, 2011 at 7:04 AM, Steven D'Aprano wrote: .. >> Whereas I read it as 'has the value' (or just 'is' ;=). > > Am I the only one who reads == as "equals"? If you are, you are the only one who reads it correctly. Consider >>> a = 2 >>> a == 2.0 True ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Remove some extraneous parentheses and swap the comparison order to
On Wed, 08 Jun 2011 21:04:48 +1000, Steven D'Aprano wrote: > Terry Reedy wrote: > > On 6/7/2011 5:35 PM, David Malcolm wrote: > > > >> I know that this style is unpopular, but if it helps, try mentally > >> pronouncing "==" in C as "is the value of". > >> > >> In this example, when I read that line, my mind is thinking: > >> > >> "if 'u' is the value of typecode" > >> > >> After ~12 years of doing this, it comes naturally. I appreciate that > >> this may come across as weird though :) > > > > Whereas I read it as 'has the value' (or just 'is' ;=). > > > Am I the only one who reads == as "equals"? No :) Especially considering that Python actually has an 'is' operator -- R. David Murray http://www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can we improve support for abstract base classes with desciptors
On Thu, Jun 9, 2011 at 1:01 AM, Darren Dale wrote:
[snip excellent analysis of the problem]
I have some suggestions regarding a few details of your current code,
but your basic proposal looks sound to me.
I would tweak __new__ along the following lines though:
def __new__(mcls, name, bases, namespace):
cls = super().__new__(mcls, name, bases, namespace)
# Compute set of abstract method names
# CHANGE 1: refactor descriptor and abstract method scan to
happen in a single pass
def is_descriptor(value):
return (hasattr(value, '__get__') or hasattr(value, '__set__')
or hasattr(value, '__delete__'))
def is_abstract(value):
return getattr(value, "__isabstractmethod__", False)
def get_abstract_names_for_item(item):
name, value = item
if is_abstract(value):
return [name]
elif is_descriptor(value):
# CHANGE 2: Deliberately ignore descriptors on the
descriptor objects
# CHANGE 3: Use new-style string formatting
return ['{}.{}'.format(name, attr) for attr in
dir(value) if is_abstract(getattr(value, attr))]
return []
def get_abstract_names(ns):
names = []
for item in ns.items():
names.extend(get_abstract_names_for_item(item))
return names
abstract_names = get_abstract_names(namespace.items())
for base in bases:
for name in getattr(base, "__abstractmethods__", ()):
# CHANGE 4: Using rpartition better tolerates weird
naming in the metaclass
# (weird naming in descriptors will still blow up in
the earlier search for abstract names)
descr_name, is_descr, attr = name.rpartition('.')
if is_descr:
# base class identified a descriptor abstract method:
descr = getattr(cls, descr_name, None)
val = getattr(descr, attr, None)
else:
val = getattr(cls, name, None)
if val is None or is_abstract(val):
abstract_names.append(name)
cls.__abstractmethods__ = frozenset(abstract_names)
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The socket HOWTO
On Tue, Jun 7, 2011 at 10:37 AM, Eli Bendersky wrote: > > Just be careful not to reproduce http://www.apress.com/9781590593714 :-) > These things tend to get out of hand very quickly. You say that like it's a bad thing. The first few chapters of that would make a great replacement for the howto. Geremy Condra ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] fatal error callback issue
I'm writing in regards to http://bugs.python.org/issue1195571 I'm embedding Python in my application and ran into a need for this functionality. I wrote a similar patch myself, and was about to submit it. When I searched for similar issues I found that this one has been available since 2005. I'd really like to help get this patch approved and integrated into the python sources. I'm sure many other python embedders have run into this in the past - it's a legitimate concern for any process which is not 100% hosted in the Python world. Thanks. Tom. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can we improve support for abstract base classes with desciptors
On Wed, Jun 8, 2011 at 11:55 AM, Nick Coghlan wrote: > On Thu, Jun 9, 2011 at 1:01 AM, Darren Dale wrote: > [snip excellent analysis of the problem] > > I have some suggestions regarding a few details of your current code, > but your basic proposal looks sound to me. > > I would tweak __new__ along the following lines though: [snip] Thank you, I agree. Concerning the following block: > def get_abstract_names(ns): > names = [] > for item in ns.items(): > names.extend(get_abstract_names_for_item(item)) > return names > > abstract_names = get_abstract_names(namespace.items()) That should be "get_abstract_names(namespace)", since ns.items() gets called again in the for loop. I think the get_abstract_names function isn't needed though, since it is only ever called that one time. Any reason not replace the above block with:: abstract_names = [] for item in namespace.items(): abstract_names.extend(get_abstract_names_for_item(item)) > for base in bases: > for name in getattr(base, "__abstractmethods__", ()): > # CHANGE 4: Using rpartition better tolerates weird > naming in the metaclass > # (weird naming in descriptors will still blow up in > the earlier search for abstract names) Could you provide an example of weird naming? Darren ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (2.7): Merge
Le jeudi 09 juin 2011 à 02:30 +0200, brian.curtin a écrit : > http://hg.python.org/cpython/rev/f1509fc75435 > changeset: 70715:f1509fc75435 > branch: 2.7 > parent: 70661:6e7a98cfcfab > user:Brian Curtin > date:Wed Jun 08 19:29:53 2011 -0500 > summary: > Merge FYI this commit has only one parent, it's not a merge. It should have a changelog describing the patch. ... whereas the following commit has two parents, it's a merge: > http://hg.python.org/cpython/rev/567f30527913 > changeset: 70714:567f30527913 > parent: 70712:964d0d65a2a9 > parent: 70713:88e318166eaf > user:Brian Curtin > date:Wed Jun 08 18:43:57 2011 -0500 > summary: > Fix #11583. Changed os.path.isdir to use GetFileAttributes (...) You can use "merge", "Merge 3.2", or other similir changelog. I prefer to use "(Merge 3.2)Fix #11583. Changed os.path._isdir ..." : copy/paste the changelog of the original commit with "(Merge 3.2) " prefix. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] fatal error callback issue
On 6/8/2011 3:30 PM, Tom Whittock wrote: I'm writing in regards to http://bugs.python.org/issue1195571 I'm embedding Python in my application and ran into a need for this functionality. I wrote a similar patch myself, and was about to submit it. When I searched for similar issues I found that this one has been available since 2005. I'd really like to help get this patch approved and integrated into the python sources. I'm sure many other python embedders have run into this in the past - it's a legitimate concern for any process which is not 100% hosted in the Python world. Add a comment like this to the issue itself. Also review the most recent patch, considering Victor's comments. Is it better than yours? Can you improve it? Can you test it? -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can we improve support for abstract base classes with desciptors
On Thu, Jun 9, 2011 at 8:51 AM, Darren Dale wrote: > That should be "get_abstract_names(namespace)", since ns.items() gets > called again in the for loop. I think the get_abstract_names function > isn't needed though, since it is only ever called that one time. Any > reason not replace the above block with:: > > abstract_names = [] > for item in namespace.items(): > abstract_names.extend(get_abstract_names_for_item(item)) Nope, inlining that part makes sense. >> for base in bases: >> for name in getattr(base, "__abstractmethods__", ()): >> # CHANGE 4: Using rpartition better tolerates weird >> naming in the metaclass >> # (weird naming in descriptors will still blow up in >> the earlier search for abstract names) > > Could you provide an example of weird naming? >>> class C(object): ... pass ... >>> setattr(C, 'weird.name', staticmethod(int)) >>> c = C() >>> c.weird.name Traceback (most recent call last): File "", line 1, in AttributeError: 'C' object has no attribute 'weird' >>> c.weird.name Traceback (most recent call last): File "", line 1, in AttributeError: 'C' object has no attribute 'weird' >>> getattr(c, 'weird.name')() 0 This is definitely something that could legitimately be dismissed as "well, don't do that then" (particularly since similarly weird names on the descriptors will still break). However, I also prefer the way partition based code reads over split-based code, so I still like the modified version. Full tolerance for weird naming would require storing 2-tuples in __abstractmethods__ which would cause a whole new set of problems and isn't worth the hassle. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The socket HOWTO
On Wed, Jun 8, 2011 at 21:07, geremy condra wrote: > On Tue, Jun 7, 2011 at 10:37 AM, Eli Bendersky wrote: > > > > Just be careful not to reproduce http://www.apress.com/9781590593714 :-) > > These things tend to get out of hand very quickly. > > You say that like it's a bad thing. The first few chapters of that > would make a great replacement for the howto. > > Not a bad thing at all, and I'm sorry if I made it sound that way. I just meant that it may turn into a *whole book* if too many details are added. I had no intention to criticize this specific book. Frankly I didn't even read it, I just remembered that a book with this title came out recently. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.2): Fix #11583. Changed os.path.isdir to use GetFileAttributes instead of os.stat.
On 06/09/11 02:00, brian.curtin wrote: > http://hg.python.org/cpython/rev/88e318166eaf > changeset: 70713:88e318166eaf > branch: 3.2 > parent: 70700:0aa3064d1cef > user:Brian Curtin > date:Wed Jun 08 18:17:18 2011 -0500 > summary: > Fix #11583. Changed os.path.isdir to use GetFileAttributes instead of > os.stat. > > By changing to the Windows GetFileAttributes API in nt._isdir we can figure > out if the path is a directory without opening the file via os.stat. This has > the minor benefit of speeding up os.path.isdir by at least 2x for regular > files and 10-15x improvements were seen on symbolic links (which opened the > file multiple times during os.stat). Since os.path.isdir is used in > several places on interpreter startup, we get a minor speedup in startup time. > > files: > Lib/ntpath.py | 13 ++ > Misc/NEWS | 3 ++ > Modules/posixmodule.c | 37 +++ > 3 files changed, 53 insertions(+), 0 deletions(-) > > > diff --git a/Lib/ntpath.py b/Lib/ntpath.py > --- a/Lib/ntpath.py > +++ b/Lib/ntpath.py > @@ -672,3 +672,16 @@ > def sameopenfile(f1, f2): > """Test whether two file objects reference the same file""" > return _getfileinformation(f1) == _getfileinformation(f2) > + > + > +try: > +# The genericpath.isdir implementation uses os.stat and checks the mode > +# attribute to tell whether or not the path is a directory. > +# This is overkill on Windows - just pass the path to GetFileAttributes > +# and check the attribute from there. > +from nt import _isdir > +except ImportError: > +from genericpath import isdir as _isdir > + > +def isdir(path): > +return _isdir(path) Not that it matters, but ISTM that this would be faster as try: from nt import _isdir as isdir except ImportError: pass Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
