Re: [Python-Dev] Decimal & returning NotImplemented (or not)
On Tue, Mar 01, 2005 at 05:55:50PM -0500, Neil Schemenauer wrote: > On Tue, Mar 01, 2005 at 11:45:43PM +1000, Nick Coghlan wrote: > > Interesting. In that case, my other suggestion was to have raising > > NotImplementedError from a special method be the equivalent of returning > > NotImplemented (which makes life much easier for a class like Decimal which > > has an internal method for doing the type conversion). > > NotImplementedError has nothing to do with the NotImplemented > singleton. It's unfortunate about the naming. IMO, Decimal should > be returning NotImplemented instead of raising TypeError. That seems very un-pythonic to me - return an error? If you look at the patched Decimal module you'll see all sorts of contortions necessary to accomodate it, wheras if it could just raise NotImplementedError or NotImplementedTypeError it would make the code a lot cleaner. I have to say when I read the python language docs, I assumed there was a mistake in them and they meant to say "raise NotImplementedError" instead of "return NotImplemented". Why is it like that? And could it be changed (Nick Coghlan's proposal seems good to me)? -- Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & returning NotImplemented (or not)
Neil Schemenauer wrote:
IMO, Decimal should
be returning NotImplemented instead of raising TypeError.
I agree, but I'm also interested in the fact that nobody commented on this
before Python 2.4 went out. I read the reference page on numeric coercion more
times than I care to count, and still didn't register that there might be an
issue with raising the TypeError - and that's only me. Relative to people like
Raymond and Facundo, I spent comparatively little time working on Decimal :)
I think the problem arose because the natural thing to do in Python code is to
push the type inspection for operations into a common method, and have that
method raise TypeError if the other argument isn't acceptable.
Trying to convert that exception to a 'NotImplemented' return value is a pain
(it's less of a pain in C, since you're already checking for error returns
instead of using exceptions). The error return idiom is really unnatural in
Python code. Mixing the error return for the special methods with raising an
exception for non-special methods is exceedingly ugly (particularly for Decimal,
since operations through Context objects should *always* raise a TypeError for
bad arguments. If the special methods return NotImplemented instead of raising
an exception, then Context has to convert those returns to TypeErrors).
Additionally, returning NotImplemented means that direct calls to special
methods may not raise an exception in response to a bad argument. Compare:
Py> 1 .__add__("")
NotImplemented
Py> "".__add__(1)
Traceback (most recent call last):
File "", line 1, in ?
TypeError: cannot concatenate 'str' and 'int' objects
Hmm, perhaps a decorator would help here. Something like:
class OperatorTypeError(TypeError): pass
class operatormethod(object):
def __init__(self, method):
self._method = method
self._obj = None
def __get__(self, obj, type=None):
self._obj = obj
return self
def __call__(*args, **kwds):
self = args[0]
obj = self._obj
try:
if obj is None:
return self._method(*args[1:], **kwds)
return self._method(obj, *args[1:], **kwds)
except OperatorTypeError:
return NotImplemented
Then do:
Py> class C:
... def _add(self, other):
... raise OperatorTypeError
... __add__ = operatormethod(_add)
... __radd__ = __add__
...
Py> C()._add(1)
Traceback (most recent call last):
File "", line 1, in ?
File "", line 3, in _add
__main__.OperatorTypeError
Py> C().__add__(1)
NotImplemented
Py> C() + 1
Traceback (most recent call last):
File "", line 1, in ?
TypeError: unsupported operand type(s) for +: 'instance' and 'int'
Py> 1 + C()
Traceback (most recent call last):
File "", line 1, in ?
TypeError: unsupported operand type(s) for +: 'int' and 'instance'
That makes it easy to write natural Python code (raising a specific subclass of
type error), while still returning NotImplemented so the operator coercion works
correctly.
> That
> could be considered a bug but I'd file it under new functionality
> for the purposes of backporting (i.e. fix it in 2.5 only).
Given that I'm already ambivalent about changing this for 2.4, it won't take
much to convince me that this should be left to 2.5.
Particularly if we actually try to find a way to make it easier to 'do the right
thing', rather than just changing Decimal.
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.skystorm.net
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & returning NotImplemented (or not)
Nick Craig-Wood wrote: Why is it like that? Speed, I assume - if it was exception based, then *every* operation effectively gets wrapped in a try/except block. Yikes! Also, for C implementations, the error return is fairly natural. It's only when implementing operations in Python that it bites. And could it be changed (Nick Coghlan's proposal seems good to me)? Take a look at my latest suggestion using OperatorTypeError and operatormethod. (which makes it easy to put the try/catch block in place if you want it, while still leaving it out for the common case). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & returning NotImplemented (or not)
Nick Coghlan <[EMAIL PROTECTED]> writes: > Neil Schemenauer wrote: >> IMO, Decimal should >> be returning NotImplemented instead of raising TypeError. > > I agree, but I'm also interested in the fact that nobody commented on > this before Python 2.4 went out. I read the reference page on numeric > coercion more times than I care to count, and still didn't register > that there might be an issue with raising the TypeError - and that's > only me. Relative to people like Raymond and Facundo, I spent > comparatively little time working on Decimal :) > > I think the problem arose because the natural thing to do in Python > code is to push the type inspection for operations into a common > method, and have that method raise TypeError if the other argument > isn't acceptable. > > Trying to convert that exception to a 'NotImplemented' return value is > a pain (it's less of a pain in C, since you're already checking for > error returns instead of using exceptions). It seems to me that the most straightforward thing would be for _convert_other to return NotImplemented, or to split it into two functions, one that returns NotImplemented and one that raises and use the former in __methods__. This would mean a certain amount of other = _convert_other_ni(other) if other is NotImplemented: return other but this seems mild in comparisons to the contortions the module already performs in the name of correctness. Cheers, mwh PS: this beahviour seems odd already: >>> decimal.getcontext().abs(1) Traceback (most recent call last): File "", line 1, in ? File "/Users/mwh/Source/python/dist/src/Lib/decimal.py", line 2321, in abs return a.__abs__(context=self) TypeError: wrapper __abs__ doesn't take keyword arguments couldn't/shouldn't the context methods do a _convert_other (of the erroring out kind) of their arguments? -- Gullible editorial staff continues to post links to any and all articles that vaguely criticize Linux in any way. -- Reason #4 for quitting slashdot today, from http://www.cs.washington.edu/homes/klee/misc/slashdot.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] assigning a custom mapping type to __dict__
I think the behavior when supplying a custom mapping type to __dict__
is a little confusing. I'd like to supply a patch, but I'm not sure
what the preferred solution is.
Let me explain the problem a little first.
For a custom mapping type that does not inherit from dict, Python
gives a nice error message when it's assigned to __dict__:
py> class M(object):
... def __getitem__(self, key):
... return 42
...
py> class O(object):
...pass
...
py> o = O()
py> o.__dict__ = M()
Traceback (most recent call last):
File "", line 1, in ?
TypeError: __dict__ must be set to a dictionary
However, a subclass of dict will still be accepted, but has some
confusing behavior -- a redefined __getitem__ is not actually invoked:
py> class M(dict):
... def __getitem__(self, key):
... return 42
...
py> class O(object):
...pass
...
py> o = O()
py> o.__dict__ = M()
py> o.a
Traceback (most recent call last):
File "", line 1, in ?
AttributeError: 'O' object has no attribute 'a'
By overriding __getattribute__ in the object's class, you can get the
expected behavior:
py> class M(dict):
... def __getitem__(self, key):
... return 42
...
py> class O(object):
... def __getattribute__(self, name):
... getattribute = super(O, self).__getattribute__
... if not name.startswith('__') or not name.endswith('__'):
... try:
... return getattribute('__dict__')[name]
... except KeyError:
... raise AttributeError(name)
... return getattribute(name)
...
py> o = O()
py> o.__dict__ = M()
py> o.a
42
py> o.a = 1
py> o.a
42
So, I see this as a problem, mainly because an error (assiging a
mapping to __dict__ and expecting its methods to be called) passes
silently. I guess the first question is, do others agree this is also
an problem?
If so, I see a few solutions, and I'd like some input on which is most
desirable.
(1) Raise an exception when a subclass of dict is assigned to
__dict__. This is probably the simplest solution (I assume it's
basically just changing a PyDict_Check to a PyDict_CheckExact), and it
keeps the error from passing silently, but it is
backwards-incompatible. (Classes like my last versions of M and O
above will be broken.)
(2) Allow sublcasses of dict to be assigned to __dict__, but change
the implementation to call the subclass methods, not the dict methods.
I don't know exactly what needs to be changed to make this work.
(3) Allow any mapping type to be assigned to __dict__, and change the
implementation to use Py_Mapping* calls instead.
I don't know the implementation well enough to know the consequences
of the last two, so I'm hoping for some feedback here.
Thanks!
Steve
--
You can wordify anything if you just verb it.
--- Bucky Katt, Get Fuzzy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: OK, time to retire
Brett C. wrote: I have decided that I am going to stop doing the python-dev Summaries > after PyCon; the Summary covering the last half of March 2005 will be > it for me. I (as well as most, I'd guess) have enjoyed your voice in the summaries. Thanks for a great series of summaries. Perhaps your final summary could be a personal view of PyCon for those of us unable to get there. If you make no more contribution to Python than you have so far, you will have done us a great service. Hip-hip-hooray-ly y'rs --Scott David Daniels [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & returning NotImplemented (or not)
> Nick Craig-Wood wrote: > > Why is it like that? Nick Coghlan replied: > Speed, I assume - if it was exception based, then *every* operation > effectively > gets wrapped in a try/except block. Yikes! No, the reason is that if we did this with exceptions, it would be liable to mask errors; an exception does not necessarily originate immediately with the code you invoked, it could have been raised by something else that was invoked by that code. The special value NotImplemented must pretty much originate directly in the invoked code, since the implementation guarantees that e.g. a+b can never *return* NotImplemented: if a.__add__(b) and b.__radd__(a) both return NotImplemented, TypeError is raised. (Then why is StopIteration an exception instead of a special value, you may ask? That's because we didn't want to have a value that you could stick in a list to stop iteration over the list prematurely, which would cause problems similar to null bytes in C strings. For binary operators this isn't a problem, since binary operators (at least by convention) always return an object that itself supports the same binary operator that produced it, and NotImplemented itself doesn't implement any binary operators.) > Also, for C implementations, the error return is fairly natural. It's only > when > implementing operations in Python that it bites. You went on to great lengths later about how it's less natural in Python to return an error value, but I disagree. I've written a lot of code like this and it's quite natural to write things like this: def __add__(self, other): if not isinstance(other, ThisClass): return NotImplemented return ...implementation of self + other... If you want to factor out the type check, it makes just as much sense to define a Boolean function: def __add__(self, other): if not self.acceptableArgument(other): return NotImplemented ...etc... I'm guessing this wasn't caught before 2.4 was released because most people reviewing the code were more interested in correct computations than into correct integration in the Python framework. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
