Sorry, was too quick. For immutable types __iop__ may not exist and then the fallback machinery should work normally using NotImplemented. But if __iop__ exists it can choose not to allow __rop__, because the type would presumably change. This is probably more predictable. I don't even know if the byte code interpreter looks for Not implemented from __iop__. On Nov 3, 2014 9:00 AM, "Guido van Rossum" <gu...@python.org> wrote:
> Not those. > On Nov 3, 2014 8:56 AM, "Antoine Pitrou" <solip...@pitrou.net> wrote: > >> On Mon, 3 Nov 2014 08:48:07 -0800 >> Guido van Rossum <gu...@python.org> wrote: >> > Gotta be brief, but NotImplemented is for all binary ops. >> >> Even in-place ops? >> >> Regards >> >> Antoine. >> >> >> > Power may be an >> > exception because it's ternary? >> > On Nov 3, 2014 8:08 AM, "Brett Cannon" <br...@python.org> wrote: >> > >> > > >> > > >> > > On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman <et...@stoneleaf.us> >> wrote: >> > > >> > >> Just to be clear, this is about NotImplemented, not >> NotImplementedError. >> > >> >> > >> tl;dr When a binary operation fails, should an exception be raised >> or >> > >> NotImplemented returned? >> > >> >> > > >> > > The docs for NotImplemented suggest it's only for rich comparison >> methods >> > > and not all binary operators: >> > > https://docs.python.org/3/library/constants.html#NotImplemented . But >> > > then had I not read that I would have said all binary operator methods >> > > should return NotImplemented when the types are incompatible. >> > > >> > > -Brett >> > > >> > > >> > >> >> > >> >> > >> When a binary operation in Python is attempted, there are two >> > >> possibilities: >> > >> >> > >> - it can work >> > >> - it can't work >> > >> >> > >> The main reason [1] that it can't work is that the two operands are >> of >> > >> different types, and the first type does not know >> > >> how to deal with the second type. >> > >> >> > >> The question then becomes: how does the first type tell Python that >> it >> > >> cannot perform the requested operation? The most >> > >> obvious answer is to raise an exception, and TypeError is a good >> > >> candidate. The problem with the exception raising >> > >> approach is that once an exception is raised, Python doesn't try >> anything >> > >> else to make the operation work. >> > >> >> > >> What's wrong with that? Well, the second type might know how to >> perform >> > >> the operation, and in fact that is why we have >> > >> the reflected special methods, such as __radd__ and __rmod__ -- but >> if >> > >> the first type raises an exception the __rxxx__ >> > >> methods will not be tried. >> > >> >> > >> Okay, how can the first type tell Python that it cannot do what is >> > >> requested, but to go ahead and check with the second >> > >> type to see if it does? That is where NotImplemented comes in -- if >> a >> > >> special method (and only a special method) >> > >> returns NotImplemented then Python will check to see if there is >> anything >> > >> else it can do to make the operation succeed; >> > >> if all attempts return NotImplemented, then Python itself will raise >> an >> > >> appropriate exception [2]. >> > >> >> > >> In an effort to see how often NotImplemented is currently being >> returned >> > >> I crafted a test script [3] to test the types >> > >> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict, >> > >> deque, and OrderedDict with the operations for >> > >> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__, >> > >> __ilshift__, __imod__, __imul__, __ior__, __ipow__, >> > >> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__, >> > >> __mul__, __or__, __pow__, __rshift__, __sub__, >> > >> __truediv__, and __xor__. >> > >> >> > >> Here are the results of the 275 tests: >> > >> ------------------------------------------------------------ >> > >> -------------------- >> > >> testing control... >> > >> >> > >> ipow -- Exception <unsupported operand type(s) for ** or pow(): >> 'Control' >> > >> and 'subtype'> raised >> > >> errors in Control -- misunderstanding or bug? >> > >> >> > >> testing types against a foreign class >> > >> >> > >> iadd(Counter()) -- Exception <'SomeOtherClass' object has no >> attribute >> > >> 'items'> raised instead of TypeError >> > >> iand(Counter()) -- NotImplemented not returned, TypeError not raised >> > >> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute >> > >> 'items'> raised instead of TypeError >> > >> isub(Counter()) -- Exception <'SomeOtherClass' object has no >> attribute >> > >> 'items'> raised instead of TypeError >> > >> >> > >> >> > >> testing types against a subclass >> > >> >> > >> mod(str()) -- NotImplemented not returned, TypeError not raised >> > >> >> > >> iadd(Counter()) -- Exception <'subtype' object has no attribute >> 'items'> >> > >> raised (should have worked) >> > >> iand(Counter()) -- NotImplemented not returned, TypeError not raised >> > >> ior(Counter()) -- Exception <'subtype' object has no attribute >> 'items'> >> > >> raised (should have worked) >> > >> isub(Counter()) -- Exception <'subtype' object has no attribute >> 'items'> >> > >> raised (should have worked) >> > >> ------------------------------------------------------------ >> > >> -------------------- >> > >> >> > >> Two observations: >> > >> >> > >> - __ipow__ doesn't seem to behave properly in the 3.x line (that >> error >> > >> doesn't show up when testing against 2.7) >> > >> >> > >> - Counter should be returning NotImplemented instead of raising an >> > >> AttributeError, for three reasons [4]: >> > >> - a TypeError is more appropriate >> > >> - subclasses /cannot/ work with the current implementation >> > >> - __iand__ is currently a silent failure if the Counter is >> empty, >> > >> and the other operand should trigger a failure >> > >> >> > >> Back to the main point... >> > >> >> > >> So, if my understanding is correct: >> > >> >> > >> - NotImplemented is used to signal Python that the requested >> operation >> > >> could not be performed >> > >> - it should be used by the binary special methods to signal type >> > >> mismatch failure, so any subclass gets a chance to work. >> > >> >> > >> Is my understanding correct? Is this already in the docs somewhere, >> and >> > >> I just missed it? >> > >> >> > >> -- >> > >> ~Ethan~ >> > >> >> > >> [1] at least, it's the main reason in my code >> > >> [2] usually a TypeError, stating either that the operation is not >> > >> supported, or the types are unorderable >> > >> [3] test script at the end >> > >> [4] https://bugs.python.org/issue22766 [returning NotImplemented was >> > >> rejected] >> > >> >> > >> -- 8< ------------------------------------------------------------ >> > >> ---------------- >> > >> from collections import Counter, defaultdict, deque, OrderedDict >> > >> from fractions import Fraction >> > >> from decimal import Decimal >> > >> from enum import Enum >> > >> import operator >> > >> import sys >> > >> >> > >> py_ver = sys.version_info[:2] >> > >> >> > >> types = ( >> > >> bytes, bytearray, str, dict, list, tuple, >> > >> Enum, Counter, defaultdict, deque, OrderedDict, >> > >> ) >> > >> numeric_types = int, float, Decimal, Fraction >> > >> >> > >> operators = ( >> > >> '__add__', '__and__', '__floordiv__', >> > >> '__iadd__', '__iand__', '__ifloordiv__', '__ilshift__', >> > >> '__imod__', '__imul__', '__ior__', '__ipow__', >> > >> '__irshift__', '__isub__', '__itruediv__', '__ixor__', >> > >> '__lshift__', '__mod__', '__mul__', >> > >> '__or__', '__pow__', '__rshift__', '__sub__', '__truediv__', >> > >> '__xor__', >> > >> ) >> > >> >> > >> if py_ver >= (3, 0): >> > >> operators += ('__gt__', '__ge__', '__le__','__lt__') >> > >> >> > >> ordered_reflections = { >> > >> '__le__': '__ge__', >> > >> '__lt__': '__gt__', >> > >> '__ge__': '__le__', >> > >> '__gt__': '__lt__', >> > >> } >> > >> >> > >> >> > >> # helpers >> > >> >> > >> class SomeOtherClass: >> > >> """" >> > >> used to test behavior when a different type is passed in to the >> > >> special methods >> > >> """ >> > >> def __repr__(self): >> > >> return 'SomeOtherClass' >> > >> some_other_class = SomeOtherClass() >> > >> >> > >> class MainClassHandled(Exception): >> > >> """ >> > >> called by base class if both operands are of type base class >> > >> """ >> > >> >> > >> class SubClassCalled(Exception): >> > >> """ >> > >> called by reflected operations for testing >> > >> """ >> > >> >> > >> def create_control(test_op): >> > >> def _any(self, other): >> > >> if not type(other) is self.__class__: >> > >> return NotImplemented >> > >> raise MainClassHandled >> > >> class Control: >> > >> "returns NotImplemented when other object is not supported" >> > >> _any.__name__ = op >> > >> setattr(Control, test_op, _any) >> > >> return Control() >> > >> >> > >> def create_subtype(test_op, base_class=object): >> > >> def _any(*a): >> > >> global subclass_called >> > >> subclass_called = True >> > >> raise SubClassCalled >> > >> class subtype(base_class): >> > >> __add__ = __sub__ = __mul__ = __truediv__ = __floordiv__ = >> _any >> > >> __mod__ = __divmod__ = __pow__ = __lshift__ = __rshift__ = >> _any >> > >> __and__ = __xor__ = __or__ = _any >> > >> __radd__ = __rsub__ = __rmul__ = __rtruediv__ = >> __rfloordiv__ = >> > >> _any >> > >> __rmod__ = __rdivmod__ = __rpow__ = __rlshift__ = >> __rrshift__ = >> > >> _any >> > >> __rand__ = __rxor__ = __ror__ = _any >> > >> __le__ = __lt__ = __gt__ = __ge__ = _any >> > >> if issubclass(subtype, (bytes, bytearray)): >> > >> value = b'hello' >> > >> elif issubclass(subtype, str): >> > >> value = 'goodbye' >> > >> elif issubclass(subtype, (list, tuple)): >> > >> value = (1, 2, 3) >> > >> elif issubclass(subtype, (int, float, Decimal, Fraction)): >> > >> value = 42 >> > >> else: >> > >> # ignore value >> > >> return subtype() >> > >> return subtype(value) >> > >> >> > >> >> > >> # test exceptions >> > >> >> > >> # control against some other class >> > >> print('testing control...\n') >> > >> errors = False >> > >> for op in operators: >> > >> control = create_control(op) >> > >> op = getattr(operator, op) >> > >> try: >> > >> op(control, some_other_class) >> > >> except TypeError: >> > >> # the end result of no method existing, or each method >> called >> > >> returning >> > >> # NotImplemented because it does not know how to perform the >> > >> requested >> > >> # operation between the two types >> > >> pass >> > >> except Exception as exc: >> > >> errors = True >> > >> print('%s(%s()) -- Exception <%s> raised instead of >> TypeError' % >> > >> (op.__name__, test_type.__name__, exc)) >> > >> else: >> > >> errors = True >> > >> print('Control -- TypeError not raised for op %r' % op) >> > >> if errors: >> > >> print('errors in Control -- misunderstanding or bug?\n') >> > >> >> > >> # control against a subclass >> > >> errors = False >> > >> for op in operators: >> > >> subclass_called = False >> > >> control = create_control(op) >> > >> subtype = create_subtype(op, control.__class__) >> > >> op = getattr(operator, op) >> > >> try: >> > >> op(control, subtype) >> > >> except SubClassCalled: >> > >> # if the control class properly signals that it doesn't >> know how >> > >> to >> > >> # perform the operation, of if Python notices that a >> reflected >> > >> # operation exists, we get here (which is good) >> > >> pass >> > >> except MainClassHandled: >> > >> errors = True >> > >> print('Control did not yield to subclass for op %r' % op) >> > >> except Exception as exc: >> > >> if subclass_called: >> > >> # exception was subverted to something more appropriate >> (like >> > >> # unorderable types) >> > >> pass >> > >> errors = True >> > >> print('%s -- Exception <%s> raised' % >> > >> (op.__name__, exc)) >> > >> else: >> > >> errors = True >> > >> print('Control -- op %r appears to have succeeded (it >> should not >> > >> have)' % op) >> > >> if errors: >> > >> print('errors in Control -- misunderstanding or bug?\n') >> > >> >> > >> >> > >> # tests >> > >> print('testing types against a foreign class\n') >> > >> for test_type in types + numeric_types: >> > >> errors = False >> > >> for op in operators: >> > >> op = getattr(operator, op) >> > >> try: >> > >> op(test_type(), some_other_class) >> > >> except TypeError: >> > >> pass >> > >> except Exception as exc: >> > >> errors = True >> > >> print('%s(%s()) -- Exception <%s> raised instead of >> > >> TypeError' % >> > >> (op.__name__, test_type.__name__, exc)) >> > >> else: >> > >> print('%s(%s()) -- NotImplemented not returned, >> TypeError >> > >> not raised' % >> > >> (op.__name__, test_type.__name__)) >> > >> if errors: >> > >> print() >> > >> >> > >> print() >> > >> >> > >> # test subclasses >> > >> print('testing types against a subclass\n') >> > >> for test_type in types: >> > >> errors = False >> > >> for op in operators: >> > >> subclass_called = False >> > >> if not test_type.__dict__.get(op): >> > >> continue >> > >> subclass = create_subtype(op, test_type) >> > >> op = getattr(operator, op) >> > >> try: >> > >> if test_type is str: >> > >> op('%s', subtype) >> > >> else: >> > >> op(test_type(), subtype) >> > >> except SubClassCalled: >> > >> # expected, ignore >> > >> pass >> > >> except Exception as exc: >> > >> if subclass_called: >> > >> # exception raised by subclass was changed >> > >> pass >> > >> errors = True >> > >> print('%s(%s()) -- Exception <%s> raised (should have >> > >> worked)' % >> > >> (op.__name__, test_type.__name__, exc)) >> > >> else: >> > >> errors = True >> > >> print('%s(%s()) -- NotImplemented not returned, >> TypeError >> > >> not raised' % >> > >> (op.__name__, test_type.__name__)) >> > >> if errors: >> > >> print() >> > >> for test_type in numeric_types: >> > >> errors = False >> > >> for op in operators: >> > >> subclass_called = False >> > >> if not test_type.__dict__.get(op): >> > >> continue >> > >> subtype = create_subtype(op, test_type) >> > >> op = getattr(operator, op) >> > >> try: >> > >> op(test_type(), subtype) >> > >> except SubClassCalled: >> > >> # expected, ignore >> > >> pass >> > >> except Exception as exc: >> > >> if subclass_called: >> > >> # exception raised by subclass was changed >> > >> pass >> > >> errors = True >> > >> print('%s(%s()) -- Exception <%s> raised (should have >> > >> worked)' % >> > >> (op.__name__, test_type.__name__, exc)) >> > >> else: >> > >> errors = True >> > >> print('%s(%s)) -- NotImplemented not returned' % >> > >> (op.__name__, test_type.__name__)) >> > >> if errors: >> > >> print() >> > >> -- 8< ------------------------------------------------------------ >> > >> ---------------- >> > >> _______________________________________________ >> > >> Python-Dev mailing list >> > >> Python-Dev@python.org >> > >> https://mail.python.org/mailman/listinfo/python-dev >> > >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ >> > >> brett%40python.org >> > >> >> > > >> > > _______________________________________________ >> > > Python-Dev mailing list >> > > Python-Dev@python.org >> > > https://mail.python.org/mailman/listinfo/python-dev >> > > Unsubscribe: >> > > https://mail.python.org/mailman/options/python-dev/guido%40python.org >> > > >> > > >> > >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/guido%40python.org >> >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
