Case study: library class inheritance with property declarations
Hi all,
It's possible that I'm missing the obvious -- I've been up for over 24
hours and I'm most likely dehydrated from mass coffee intake, but I
figure many people in similar circumstances will be searching
comp.lang.python one day, so here goes!
class LibraryClass(object):
"""
A class whose implementation is out of my hands.
"""
def __init__(self):
"""
Follows good dynamic-language form and binds all instance
variables at initialization time.
"""
# Omitted: complex initialization functionality.
self.useful_attr = None
class MyInheritedClass(LibraryClass):
"""
My refinement of the functionality offered by the LibraryClass. I
now want the instance to initialize with a reference to an
external
object, and the useful_attr defined in the superclass will now
reference an attribute of that external object via fget.
Changing the attribute of the external object has undefined
behavior, so I want to omit an fset in the property declaration;
however, I have to provide some way for the superclass to
initialize useful_attr -- I can't change the superclass' code, as
it
resides in a library that is out of my hands.
"""
def __init__(self, external_obj):
LibraryClass.__init__(self)
self._external_obj = external_obj
def get_useful_attr(self):
return self._external_obj.proxy_useful_attr
useful_attr = property(fget=get_useful_attr)
def test():
class _Fake(object):
pass
external_obj = _Fake()
external_obj.proxy_useful_attr = 12
spam = MyInheritedClass(external_obj)
if __name__ == '__main__':
test()
EOF
If you're one of those people who doesn't like laboriously reading
contrived examples (elitists ;) I'll boil it down for you: Library
class initializes some attribute, but derived class wants to eliminate
fsets for said attribute. As a result, our ideal solution
Of course, this means that the derived class will raise an error in
some circumstances where the base class wouldn't (when you're setting
that attribute), but one can assume that the inheritance is
worthwhile.
How do I come up with silly solutions to circumvent this? Let me count
the ways...
1. So-and-so: make an fset that does nothing. This ignores (what
should be) errors in code that uses MyInheritedClass in an attempt to
accommodate a useless statement in the base class -- surely non-ideal.
2. The ugly one: since you can probably view the library, copy and
paste the complex initialization functionality in the above, but leave
out the bad statement. This not only forfeits the ideals of
inheritance, but makes you totally incompatible with future library
changes.
3. Cheerleader: Pure evil. On top of the ugliness of 2, you assume
that across library revisions the indenting won't change and that the
troublesome statement will remain on the same line, and pull off one
of these babies:
def super_evil_test():
from inspect import getsourcelines
exec(''.join([line[4:] for line in
getsourcelines(LibraryClass.__init__)[0][:-1]]))
LibraryClass.__init__ = __init__
test() # Passes, but several angels no longer get their wings
Totally kidding, everybody! I hope Guido doesn't read this thread...
And this concludes the sleep deprived rambling that follows the
somewhat interesting case in point. Thoughts?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Case study: library class inheritance with property declarations
On Aug 2, 6:49 am, [EMAIL PROTECTED] wrote:
> Hi all,
>
> It's possible that I'm missing the obvious -- I've been up for over 24
> hours and I'm most likely dehydrated from mass coffee intake, but I
> figure many people in similar circumstances will be searching
> comp.lang.python one day, so here goes!
>
> class LibraryClass(object):
>
> """
> A class whose implementation is out of my hands.
> """
>
> def __init__(self):
> """
> Follows good dynamic-language form and binds all instance
> variables at initialization time.
> """
> # Omitted: complex initialization functionality.
> self.useful_attr = None
>
> class MyInheritedClass(LibraryClass):
>
> """
> My refinement of the functionality offered by the LibraryClass. I
> now want the instance to initialize with a reference to an
> external
> object, and the useful_attr defined in the superclass will now
> reference an attribute of that external object via fget.
>
> Changing the attribute of the external object has undefined
> behavior, so I want to omit an fset in the property declaration;
> however, I have to provide some way for the superclass to
> initialize useful_attr -- I can't change the superclass' code, as
> it
> resides in a library that is out of my hands.
> """
>
> def __init__(self, external_obj):
> LibraryClass.__init__(self)
> self._external_obj = external_obj
>
> def get_useful_attr(self):
> return self._external_obj.proxy_useful_attr
>
> useful_attr = property(fget=get_useful_attr)
>
> def test():
> class _Fake(object):
> pass
> external_obj = _Fake()
> external_obj.proxy_useful_attr = 12
> spam = MyInheritedClass(external_obj)
>
> if __name__ == '__main__':
> test()
> EOF
>
> If you're one of those people who doesn't like laboriously reading
> contrived examples (elitists ;) I'll boil it down for you: Library
> class initializes some attribute, but derived class wants to eliminate
> fsets for said attribute. As a result, our ideal solution
>
> Of course, this means that the derived class will raise an error in
> some circumstances where the base class wouldn't (when you're setting
> that attribute), but one can assume that the inheritance is
> worthwhile.
>
> How do I come up with silly solutions to circumvent this? Let me count
> the ways...
>
> 1. So-and-so: make an fset that does nothing. This ignores (what
> should be) errors in code that uses MyInheritedClass in an attempt to
> accommodate a useless statement in the base class -- surely non-ideal.
> 2. The ugly one: since you can probably view the library, copy and
> paste the complex initialization functionality in the above, but leave
> out the bad statement. This not only forfeits the ideals of
> inheritance, but makes you totally incompatible with future library
> changes.
> 3. Cheerleader: Pure evil. On top of the ugliness of 2, you assume
> that across library revisions the indenting won't change and that the
> troublesome statement will remain on the same line, and pull off one
> of these babies:
>
> def super_evil_test():
> from inspect import getsourcelines
> exec(''.join([line[4:] for line in
> getsourcelines(LibraryClass.__init__)[0][:-1]]))
> LibraryClass.__init__ = __init__
> test() # Passes, but several angels no longer get their wings
>
> Totally kidding, everybody! I hope Guido doesn't read this thread...
>
> And this concludes the sleep deprived rambling that follows the
> somewhat interesting case in point. Thoughts?
I'm sorry -- the solution was not /enough/ coffee. Got another cup and
sat down with the type/class unification doc, and found this thought-
stimulating portion:
http://www.python.org/download/releases/2.2/descrintro/#property
If you want to override the __get__ operation for properties when used
as a class attribute, you can subclass property - it is a new-style
type itself - to extend its __get__ method, or you can define a
descriptor type from scratch by creating a new-style class that
defines __get__, __set__ and __delete__ methods.
...
The get method won't be called when the property is accessed as a
class attribute (C.x) instead of as an instance attribute (C().x).
Seeing as how property is just a wrapper class, we don't need to
declare it in the class body, though it /is/ the convention and the
way it's done in all the docs I've seen. We fix our inherited class to
be the following:
[snip]
class MyInheritedClass(LibraryClass):
"""
My refinement of the functionality offered by the LibraryClass. I
now want the instance to initialize with a reference to an
external
object, and the useful_attr defined in the superclass will now
reference an attribute of that external object via fget.
Changing the attribute of the external object has undefined
behavior, so I want to omit an fset in the property declaration;
howe
Re: Case study: library class inheritance with property declarations
On Aug 2, 7:08 am, [EMAIL PROTECTED] wrote:
> On Aug 2, 6:49 am, [EMAIL PROTECTED] wrote:
>
>
>
> > Hi all,
>
> > It's possible that I'm missing the obvious -- I've been up for over 24
> > hours and I'm most likely dehydrated from mass coffee intake, but I
> > figure many people in similar circumstances will be searching
> > comp.lang.python one day, so here goes!
>
> > class LibraryClass(object):
>
> > """
> > A class whose implementation is out of my hands.
> > """
>
> > def __init__(self):
> > """
> > Follows good dynamic-language form and binds all instance
> > variables at initialization time.
> > """
> > # Omitted: complex initialization functionality.
> > self.useful_attr = None
>
> > class MyInheritedClass(LibraryClass):
>
> > """
> > My refinement of the functionality offered by the LibraryClass. I
> > now want the instance to initialize with a reference to an
> > external
> > object, and the useful_attr defined in the superclass will now
> > reference an attribute of that external object via fget.
>
> > Changing the attribute of the external object has undefined
> > behavior, so I want to omit an fset in the property declaration;
> > however, I have to provide some way for the superclass to
> > initialize useful_attr -- I can't change the superclass' code, as
> > it
> > resides in a library that is out of my hands.
> > """
>
> > def __init__(self, external_obj):
> > LibraryClass.__init__(self)
> > self._external_obj = external_obj
>
> > def get_useful_attr(self):
> > return self._external_obj.proxy_useful_attr
>
> > useful_attr = property(fget=get_useful_attr)
>
> > def test():
> > class _Fake(object):
> > pass
> > external_obj = _Fake()
> > external_obj.proxy_useful_attr = 12
> > spam = MyInheritedClass(external_obj)
>
> > if __name__ == '__main__':
> > test()
> > EOF
>
> > If you're one of those people who doesn't like laboriously reading
> > contrived examples (elitists ;) I'll boil it down for you: Library
> > class initializes some attribute, but derived class wants to eliminate
> > fsets for said attribute. As a result, our ideal solution
>
> > Of course, this means that the derived class will raise an error in
> > some circumstances where the base class wouldn't (when you're setting
> > that attribute), but one can assume that the inheritance is
> > worthwhile.
>
> > How do I come up with silly solutions to circumvent this? Let me count
> > the ways...
>
> > 1. So-and-so: make an fset that does nothing. This ignores (what
> > should be) errors in code that uses MyInheritedClass in an attempt to
> > accommodate a useless statement in the base class -- surely non-ideal.
> > 2. The ugly one: since you can probably view the library, copy and
> > paste the complex initialization functionality in the above, but leave
> > out the bad statement. This not only forfeits the ideals of
> > inheritance, but makes you totally incompatible with future library
> > changes.
> > 3. Cheerleader: Pure evil. On top of the ugliness of 2, you assume
> > that across library revisions the indenting won't change and that the
> > troublesome statement will remain on the same line, and pull off one
> > of these babies:
>
> > def super_evil_test():
> > from inspect import getsourcelines
> > exec(''.join([line[4:] for line in
> > getsourcelines(LibraryClass.__init__)[0][:-1]]))
> > LibraryClass.__init__ = __init__
> > test() # Passes, but several angels no longer get their wings
>
> > Totally kidding, everybody! I hope Guido doesn't read this thread...
>
> > And this concludes the sleep deprived rambling that follows the
> > somewhat interesting case in point. Thoughts?
>
> I'm sorry -- the solution was not /enough/ coffee. Got another cup and
> sat down with the type/class unification doc, and found this thought-
> stimulating portion:
>
> http://www.python.org/download/releases/2.2/descrintro/#property
> If you want to override the __get__ operation for properties when used
> as a class attribute, you can subclass property - it is a new-style
> type itself - to extend its __get__ method, or you can define a
> descriptor type from scratch by creating a new-style class that
> defines __get__, __set__ and __delete__ methods.
> ...
> The get method won't be called when the property is accessed as a
> class attribute (C.x) instead of as an instance attribute (C().x).
>
> Seeing as how property is just a wrapper class, we don't need to
> declare it in the class body, though it /is/ the convention and the
> way it's done in all the docs I've seen. We fix our inherited class to
> be the following:
>
> [snip]
> class MyInheritedClass(LibraryClass):
>
> """
> My refinement of the functionality offered by the LibraryClass. I
> now want the instance to initialize with a reference to an
> external
> object, and
Re: Case study: library class inheritance with property declarations
On Aug 2, 7:05 am, Bruno Desthuilliers <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] a écrit : > (snip) > > > Last post -- I swear. > > > I failed to realize that it's all part of an extremely well defined > > attribute resolution protocol, and handled via the descriptor > > specification. Discussing descriptors was on the TODO list for the > > type/class unification document, but there's a very fulfilling > > explanation by Raymond > > Hettinger:http://users.rcn.com/python/download/Descriptor.htm > > Also, the official doc is here:http://docs.python.org/ref/descriptors.html > > > Maybe the documentation for the property builtin should make reference > > to the descriptor specification? If nobody thinks this is silly, I'll > > submit a documentation patch in a few days. > > What to say ? As an old-time pyton user (well... 7+ years now) and > meta-programming addict, it's nothing new to me, but the fact that you > didn't realize it is certainly a clear indication that documentation is > not really up to date, so yes, submitting a patch is certainly a pretty > good idea. > > > Sorry for the spam -- hope someone besides me learns from it! > > What's to be learned is mostly that we (the whole community) should take > more care of the doc (which I never contributed FWIW, so I'm the first > one that should feel in guilt here). Actually I was wrong about this! Of course, the problem was insufficient test coverage :) The key addition: assert spam.useful_attr == 12 # FAILS!!! I failed to realize it in the last posts... When you don't declare a descriptor as a class attribute the descriptor protocol isn't invoked! The descriptor protocol looks in the /class/ attributes -- it transforms spam.useful_attr to type(spam).__dict__['useful_attr'].__get__(spam, type(spam)), the caveat being /type(spam)/. >From the doc: (http://users.rcn.com/python/download/Descriptor.htm) The implementation works through a precedence chain that gives data descriptors priority over instance variables, instance variables priority over non-data descriptors, and assigns lowest priority to __getattr__ if provided. The data descriptors are only looked up in the class attributes -- the instance attributes are returned as-is, so the above assertion fails! The addition of 'print spam.useful_attr' produces , because the portion of the descriptor protocol that invokes __get__ is does not cover instance attributes (as declared in __init__). So, we're back to the original problem, with a bit more insight. What if we declare the useful_attr property as a class attribute in the constructor after the LibraryClass initialization function? It only works once! Once you set a class attribute on the first instantiation it's obviously there the next time you try to instantiate. Now our test looks like this: def test(): class _Fake(object): pass external_obj = _Fake() external_obj.proxy_useful_attr = 12 spam = MyInheritedClass(external_obj) assert spam.useful_attr == 12 eggs = MyInheritedClass(external_obj) assert eggs.useful_attr == 12 Which provides coverage for the INCORRECT class-attribute-on- instantiation solution, since eggs won't be able to instantiate. The solution which fails this test looks like this: class MyInheritedClass(LibraryClass): # WRONG! Don't use this. def __init__(self, external_obj): LibraryClass.__init__(self) def get_useful_attr(would_be_self): # has to be unbound return would_be_self._external_obj.proxy_useful_attr MyInheritedClass.useful_attr = property(get_useful_attr) self._external_obj = external_obj At this point it seems like the only viable option is to hack getattribute to add special cases. To do this, we continue to use the unbound method getter as our argument to property, but add the getattribute hack which allows for specified instance attributes that have the descriptor protocol invoked on them: class MyInheritedClass(LibraryClass): # ... def __init__(self, external_obj): LibraryClass.__init__(self) def get_useful_attr(would_be_self): return would_be_self._external_obj.proxy_useful_attr self.useful_attr = property(fget=get_useful_attr) self._external_obj = external_obj def __getattribute__(self, attr_name): """ Hacks getattribute to allow us to return desired instance attributes with the descriptor protocol. """ instance_descrs = ['useful_attr'] if attr_name in instance_descrs and attr_name in self.__dict__: attr = self.__dict__[attr_name] return attr.__get__(self, type(self)) return object.__getattribute__(self, attr_name) This passes the above test, but obviously requires a hack -- is there a more elegant way to do it? - Chris -- http://mail.python.org/mailman/listinfo/python-list
