Re: [Python-Dev] Intricacies of calling __eq__
On 18/03/14 07:52, Maciej Fijalkowski wrote: Hi I have a question about calling __eq__ in some cases. We're thinking about doing an optimization where say: if x in d: return d[x] where d is a dict would result in only one dict lookup (the second one being constant folded away). The question is whether it's ok to do it, despite the fact that it changes the semantics on how many times __eq__ is called on x. Yes it is OK. The number of equality checks is not part of the specification of the dictionary. In fact, it differs between a 32 and 64 bit build of the same code. Consider two objects that hash to 2**33+1 and 2**34+1 respectively. On a 32 bit machine their truncated hashes are both 1, so they must be distinguished by an equality test. On a 64 bit machine their hashes are distinct and no equality check is required. Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Tue, Mar 18, 2014 at 11:19 PM, Paul Moore wrote: > On 18 March 2014 19:46, Maciej Fijalkowski wrote: >>> A question: how far away will this optimization apply? >>> >>> if x in d: >>> do_this() >>> do_that() >>> do_something_else() >>> spam = d[x] >> >> it depends what those functions do. JIT will inline them and if >> they're small, it should work (although a modification of a different >> dict is illegal, since aliasing is not proven), but at some point >> it'll give up (Note that it'll also give up with a call to C releasing >> GIL since some other thread can modify it). > > Surely in the presence of threads the optimisation is invalid anyway > as other threads can run in between each opcode (I don't know how > you'd phrase that in a way that wasn't language dependent other than > "everywhere" :-)) so > > if x in d: > # HERE > spam = d[x] > > d can be modified at HERE. (If d is a local variable, obviously the > chance that another thread has access to d is a lot lower, but do you > really do that level of alias tracking?) > > Paul not in the case of JIT that *knows* where the GIL can be released. We precisely make it not possible every few bytecodes to avoid such situations. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Tue, 18 Mar 2014 09:52:05 +0200 Maciej Fijalkowski wrote: > > We're thinking about doing an optimization where say: > > if x in d: >return d[x] > > where d is a dict would result in only one dict lookup (the second one > being constant folded away). The question is whether it's ok to do it, > despite the fact that it changes the semantics on how many times > __eq__ is called on x. I don't think it's ok. If the author of the code had wanted only one lookup, they would have written: try: return d[x] except KeyError: pass I agree that an __eq__ method with side effects is rather bad, of course. What you could do is instruct people that the latter idiom (EAFP) performs better on PyPy. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Wed, Mar 19, 2014 at 2:42 PM, Antoine Pitrou wrote: > On Tue, 18 Mar 2014 09:52:05 +0200 > Maciej Fijalkowski wrote: >> >> We're thinking about doing an optimization where say: >> >> if x in d: >>return d[x] >> >> where d is a dict would result in only one dict lookup (the second one >> being constant folded away). The question is whether it's ok to do it, >> despite the fact that it changes the semantics on how many times >> __eq__ is called on x. > > I don't think it's ok. If the author of the code had wanted only one > lookup, they would have written: > > try: > return d[x] > except KeyError: > pass > > I agree that an __eq__ method with side effects is rather bad, of > course. > What you could do is instruct people that the latter idiom (EAFP) > performs better on PyPy. > > Regards > > Antoine. I would like to point out that instructing people does not really work. Besides, other examples like this: if d[x] >= 3: d[x] += 1 don't really work. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On 03/18/2014 10:19 PM, Paul Moore wrote: Surely in the presence of threads the optimisation is invalid anyway Why? As written, the code uses no synchronization primitives to ensure that the modifications to the dict are propagated at a particular point. As a consequence, it cannot rely on the modification done at a time that coincides with execution at HERE to be immediately propagated to all threads. The optimization is as valid as a C compiler rearranging variable assignments, which also "breaks" unsychronized threaded code. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Wed, 19 Mar 2014 15:09:04 +0200 Maciej Fijalkowski wrote: > > I would like to point out that instructing people does not really > work. Besides, other examples like this: > > if d[x] >= 3: >d[x] += 1 don't really work. That's a good point. But then, perhaps PyPy should analyze the __eq__ method and decide whether it's likely to have side effects or not (the answer can be hard-coded for built-in types such as str). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Wed, Mar 19, 2014 at 3:17 PM, Antoine Pitrou wrote: > On Wed, 19 Mar 2014 15:09:04 +0200 > Maciej Fijalkowski wrote: >> >> I would like to point out that instructing people does not really >> work. Besides, other examples like this: >> >> if d[x] >= 3: >>d[x] += 1 don't really work. > > That's a good point. But then, perhaps PyPy should analyze the __eq__ > method and decide whether it's likely to have side effects or not (the > answer can be hard-coded for built-in types such as str). > > Regards > > Antoine. Ok. But then how is it valid to have "is" fast-path? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Wed, 19 Mar 2014 15:21:16 +0200 Maciej Fijalkowski wrote: > On Wed, Mar 19, 2014 at 3:17 PM, Antoine Pitrou wrote: > > On Wed, 19 Mar 2014 15:09:04 +0200 > > Maciej Fijalkowski wrote: > >> > >> I would like to point out that instructing people does not really > >> work. Besides, other examples like this: > >> > >> if d[x] >= 3: > >>d[x] += 1 don't really work. > > > > That's a good point. But then, perhaps PyPy should analyze the __eq__ > > method and decide whether it's likely to have side effects or not (the > > answer can be hard-coded for built-in types such as str). > > > > Regards > > > > Antoine. > > Ok. But then how is it valid to have "is" fast-path? What do you mean? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Wed, Mar 19, 2014 at 6:26 AM, Antoine Pitrou wrote: > On Wed, 19 Mar 2014 15:21:16 +0200 > Maciej Fijalkowski wrote: > > > On Wed, Mar 19, 2014 at 3:17 PM, Antoine Pitrou > wrote: > > > On Wed, 19 Mar 2014 15:09:04 +0200 > > > Maciej Fijalkowski wrote: > > >> > > >> I would like to point out that instructing people does not really > > >> work. Besides, other examples like this: > > >> > > >> if d[x] >= 3: > > >>d[x] += 1 don't really work. > > > > > > That's a good point. But then, perhaps PyPy should analyze the __eq__ > > > method and decide whether it's likely to have side effects or not (the > > > answer can be hard-coded for built-in types such as str). > > > > > > Regards > > > > > > Antoine. > > > > Ok. But then how is it valid to have "is" fast-path? > > What do you mean? > He means you're being unrealistically pedantic :) The number of calls to __eq__ is _already_ unpredictable, since (as Mark Shannon said) it depends among other things on the hashing algorithm and the size of the dict. If we were serious about not changing these kinds of aspects between Python implementations, we certainly shouldn't be changing them between versions of the same Python implementation -- and so we would be banned from ever changing the details of the dict type. Absurd. -- Thomas Wouters Hi! I'm an email virus! Think twice before sending your email to help me spread! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues about relative& absolute import way for Porting from python2.4 to python2.7
This mailing list is for the development *of* Python, the the *use* of
Python. Your best option for getting an answer is to ask on python-list or
python-help.
On Tue, Mar 18, 2014 at 9:50 PM, 北冰洋 wrote:
> Dear,
>
> I just wrote a sample like this:
> testPy/
> __init__.py
> client.py
> SoamFactory.c
> SoamFactory.so
> soamapi.py
> sample/testP.py
> export PYTHONPATH=$(TEST_LOCATION):$(TEST_LOCATION)/testPy
> Here's the source codes:
> *__init__.py:*
> import client
> *client.py*
> import soamapi
> class Client(soamapi.SessionCallback):
>
> def __init__(self):
> print "class Client"
> super(Client, self).__init__()
> soamapi.initialize()
>
> def create_session(self):
> sec_cb = soamapi.DefaultSecurityCallback()
> self.connection = soamapi.connect(sec_cb)
> *soamapi.py*
> import SoamFactory
>
> class ConnectionSecurityCallback(object):
> def __init__(self):
> print "class ConnectionSecurityCallback"
>
> def __del__(self):
> pass
>
> def test_P(self):
> print "test_P in class ConnectionSecurityCallback"
>
> class DefaultSecurityCallback(ConnectionSecurityCallback):
> def __init__(self):
> super(DefaultSecurityCallback, self).__init__()
> print "class DefaultSecurityCallback"
>
> def __del__(self):
> super(DefaultSecurityCallback, self).__del__()
>
> def test(self):
> print "test in class DefaultSecurityCallback"
>
> class SessionCallback(object):
> def __init__(self):
> pass
>
> def __del__(self):
> pass
>
> def connect(security_callback):
> return SoamFactory.SoamFactory_connect(security_callback)
>
> def initialize():
> SoamFactory.SoamFactory_initialize()
>
> print "call soamapi"
> *SoamFactory.c*
> #include "Python.h"
> #include "PythonLoop.c"
>
> PyObject* SOAM_API_MODULE = NULL;
> PyObject* pyModule = NULL;
>
> static PyObject* SoamFactory_initialize(PyObject* self, PyObject* args){
> PyEval_InitThreads();
> init();
> Py_RETURN_NONE;
> }
>
> static PyObject* SoamFactory_connect(PyObject* self, PyObject* args){
> PyObject* pySecCallback = NULL;
> int ok = PyArg_ParseTuple(args, "O", &pySecCallback);
> if (!ok){
> printf("parse tuple error!\n");
> Py_RETURN_NONE;
> }
> if (! PyObject_IsInstance(pySecCallback, connectSecCallback)){
> printf("pySecCallback is not an instance of
> ConnectionSecurityCallback!\n");
> Py_RETURN_NONE;
> }
> printf("Successful!\n");
> Py_RETURN_NONE;
> }
>
> static PyMethodDef SoamFactory[] = {
> {"SoamFactory_connect", SoamFactory_connect, METH_VARARGS, "connection
> function"},
> {"SoamFactory_initialize", SoamFactory_initialize, METH_VARARGS,
> "SoamFactory_initialize"},
> {NULL, NULL}
> };
>
> void initSoamFactory(){
> PyEval_InitThreads();
> Py_Initialize();
> pyModule = Py_InitModule("SoamFactory", SoamFactory);
> SOAM_API_MODULE = PyImport_ImportModule("soamapi");
> }
> *sample/testP.py*
> import testPy
>
> print ""
> submitter = testPy.client.Client()
> submitter.create_session()
> print ""
>
> When I ran it on python2.4, it worked well, and the result was
> *call soamapi*
> *after import soamapi--client.py*
> **
> *class Client*
> *class ConnectionSecurityCallback*
> *class DefaultSecurityCallback*
> *Successful!*
> **
>
> But when I ran it on python2.7, it worked beyond my expectation, the
> result was
> call soamapi
> call soamapi
>
> class Client
> class ConnectionSecurityCallback
> class DefaultSecurityCallback
> pySecCallback is not an instance of ConnectionSecurityCallback!
>
>
> I found that soamapi was imported twice, and I investigated this is
> related to absolute&relative import way. PyImport_ImportModule in
> python2.7 uses absolute import way, it will look up sys.path to get soamapi
> module, and when testP.py file import testPy module, it will find local
> module soamapi under testPy package, and binds module's name to package, as
> testPy.soamapi.
> There are two ways to correct it for python2.7, 1) Don't use import
> testPy, use import client directly to avoid using relative; 2) Use from
> __future__ import absolute_import to enable absolute import feature.
>
> But there are two Pre-conditions:
> 1) Should not modify testP.py;
> 2) Should be ran on both python2.4 and 2.7.
>
> I don't know how to fix it. Is there any official way about how to porting
> this scenario or better idea?
>
> Thanks,
> Vatel
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/opti
Re: [Python-Dev] Intricacies of calling __eq__
On Wed, 19 Mar 2014 07:09:23 -0700 Thomas Wouters wrote: > > He means you're being unrealistically pedantic :) The number of calls to > __eq__ is _already_ unpredictable, since (as Mark Shannon said) it depends > among other things on the hashing algorithm and the size of the dict. Well, I was not arguing that the optimization would be an important compatibility break. Obviously, an __eq__ which would break this optimization would be particularly dubious (and probably wrong for other reasons) :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
The docs don't seem to make any guarantee about calling __eq__ or __hash__: d[key] Return the item of d with key key. Raises a KeyError if key is not in the map. which seems to indicate that this kind of optimization should be fine. In fact I would very much like messing with the semantics of __eq__ be discouraged in the docs. Currently, the docs merely say "The truth of x==y does not imply that x!=y is false." Of course, once __eq__ and __ne__ are separately overridable nothing can be guaranteed but I've seen code where x == y and x != y do completely different things and that was not pretty. 2014-03-18 21:27 GMT-07:00 Nick Coghlan : > On 19 March 2014 11:09, Steven D'Aprano wrote: > > Although I have tentatively said I think this is okay, it is a change in > > actual semantics of Python code: what you write is no longer what gets > > run. That makes this *very* different from changing the implementation > > of sort -- by analogy, its more like changing the semantics of > > > > a = f(x) + f(x) > > > > to only call f(x) once. I don't think you would call that an > > implementation detail, would you? Even if justified -- f(x) is a pure, > > deterministic function with no side-effects -- it would still be a > > change to the high-level behaviour of the code. > > Ah, I think this is a good alternative example. Given the stated > assumptions (true builtin dict, not modified between operations), > would we be OK with PyPI optimising the following to only do a single > dict lookup: > > a = d[x] + d[x] > > It's essentially the same optimisation as the one being discussed - in > the code *as written*, there are two lookups visible, but for any > sensible __hash__ and __eq__ implementation, they should always give > the same answer for a true builtin dict that isn't modified between > the two lookups. (and yes, PyPy has the infrastructure to enforce > those constraints safely and fall back to normal execution if they > aren't met - that ability to take opportunistic advantage of known > behaviours of particular types is one of the key things that makes it > such a powerful tool for implicit optimisations, as compared to things > like Cython and Numba, which change semantics more markedly, but also > have to be explicitly applied to specific sections of your code rather > than being applied automatically). > > I think it's certainly borderline (it's the kind of surprising > behavioural change that irritates people about C/C++ optimisers), but > I also think it's a defensible optimisation if the gain is significant > enough. > > Regards, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/antony.lee%40berkeley.edu > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'Add/Remove Programs' entry missing for 'current user only' 32-bit installations on 64-bit Windows
Am 17.03.14 22:10, schrieb Jurko Gospodnetić: > Fixing > this required manually cleaning up leftover CPython 3.4.0rc3 windows > installer registry entries. Note that the issue could not be fixed by > using the CPython 3.4.0rc3 installer as it failed due to the same problem. Did you try the 3.4.0rc3 "repair" installation? That should have undeleted (repaired) the files that you had manually deleted. > I compared MSI packaging related CPython source code (Tools/msi > folder) in 2.6.2 & 2.6.3 releases but I can not see anything suspicious > there. It could be that the observed beaviour change between those two > versions is a result of the final release packager changing his used > Windows Installer version, but I have not rebuilt CPython, and its MSI > installer to test this theory out. I believe I switched the entire Windows installation at least once in this time (from XP to Windows 7), so it's unlikely related to my personal installation. > Should I open a 'Add/Remove Programs' dialog related issue in the > CPython issue tracker? Please do. It would also good if somebody volunteered to reproduce the issue. > And possibly a separate one for making CPython > installations not fail without possible recovery if a previous CPython > installation has already been removed? Please don't. Or, at least, make it more specific. If you have manually corrupted your Python installation (by deleting essential files), you have to accept that the system installation procedures will fail. It might be possible to recover from the loss of a specific such file; for that, we would have to identify what the file is. However, the standard procedure should be to repair the installation before attempting an upgrade. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
Sorry, I definitely didn't mean to imply that this kind of optimization is valid on arbitrary subscript expressions; I thought we had restricted ourselves to talking about builtin dicts. If we do, I think this becomes a discussion about what subset of the semantics of CPython's builtins are language-specified vs implementation-dependent; my argument is that just because something results in an observable behavioral difference doesn't necessarily mean that it's a change in language semantics, if it's just a change in the implementation-dependent behavior. On Tue, Mar 18, 2014 at 9:54 PM, Stephen J. Turnbull wrote: > Kevin Modzelewski writes: > > > I think in this case, though, if we say for the sake of argument > > that the guaranteed semantics of a dictionary lookup are zero or > > I don't understand the point of that argument. It's simply false that > semantics are guaranteed, and all of the dunders might be user > functions. > > > more calls to __hash__ plus zero or more calls to __eq__, then two > > back-to-back dictionary lookups wouldn't have any observable > > differences from doing only one, unless you start to make > > assumptions about the behavior of the implementation. > > That's false. The inverse is true: you should allow the possibility of > observable differences, unless you make assumptions about the behavior > (implying there are none). > > > To me there seems to be a bit of a gap between seeing a dictionary > > lookup and knowing the exact sequence of user-functions that get > > called, far more than for example something like "a < b". > > The point here is that we *know* that there may be a user function > (the dunder that implements []) being called, and it is very hard to > determine that that function is pure. > > Your example of a caching hash is exactly the kind of impure function > that one would expect, but who knows what might be called -- there > could be a reference to a database on Mars involved (do we have a > vehicle on Mars at the moment? anyway...), which calls a pile of > Twisted code, and has latencies of many seconds. > > So Steven is precisely right -- in order to allow this optimization, > it would have to be explicitly allowed. > > Like Steven, I have no strong feeling against it, but then, I don't > have a program talking to a deep space vehicle in my near future. > Darn it! :-( > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'Add/Remove Programs' entry missing for 'current user only' 32-bit installations on 64-bit Windows
Hi. On 19.3.2014. 16:38, "Martin v. Löwis" wrote: > Am 17.03.14 22:10, schrieb Jurko Gospodnetić: >> Fixing >> this required manually cleaning up leftover CPython 3.4.0rc3 windows >> installer registry entries. Note that the issue could not be fixed >> by using the CPython 3.4.0rc3 installer as it failed due to the >> same problem. > > Did you try the 3.4.0rc3 "repair" installation? That should have > undeleted (repaired) the files that you had manually deleted. Just tested this out and repairing the same matching installation does revert the removed files. It does not reinstall pip and setuptools packages but that can be done easily by hand and so can be filed as a minor enhancement issue for the future. That at least provides a way to remove the the original installation in some way without having to manually remove its related Windows Installer registry entries. The problem with the 'Add/Remove Programs' dialog still makes this issue a bit complicated because a user not well versed in how Windows Installer tracks its installations internally would not see the installation in the 'standard place' where he is supposed to look for it, and would just be told by the new installation that 'something failed'. To use a 'repair' installation he would need to 'guess' which installation he used before and run its installer directly. When the 'Add/Remove Programs' dialog is fixed, user will be able to see & repair the installation from there, and even remove it directly if the uninstallation fails (I recall Windows asking if you want this if it can not uninstall something). I'll try to open an issue for the 'Add/Remove Programs' dialog issue soon, and I'll add a comment about the 'repair' situation there as well. Best regards, Jurko Gospodnetić ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Wed, Mar 19, 2014 at 3:26 PM, Antoine Pitrou wrote: > On Wed, 19 Mar 2014 15:21:16 +0200 > Maciej Fijalkowski wrote: > >> On Wed, Mar 19, 2014 at 3:17 PM, Antoine Pitrou wrote: >> > On Wed, 19 Mar 2014 15:09:04 +0200 >> > Maciej Fijalkowski wrote: >> >> >> >> I would like to point out that instructing people does not really >> >> work. Besides, other examples like this: >> >> >> >> if d[x] >= 3: >> >>d[x] += 1 don't really work. >> > >> > That's a good point. But then, perhaps PyPy should analyze the __eq__ >> > method and decide whether it's likely to have side effects or not (the >> > answer can be hard-coded for built-in types such as str). >> > >> > Regards >> > >> > Antoine. >> >> Ok. But then how is it valid to have "is" fast-path? > > What do you mean? > I mean that dict starts with "is" before calling __eq__, so the number of calls to __eq__ can as well be zero. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Wed, Mar 19, 2014 at 8:38 AM, Kevin Modzelewski wrote: > Sorry, I definitely didn't mean to imply that this kind of optimization is > valid on arbitrary subscript expressions; I thought we had restricted > ourselves to talking about builtin dicts. If we do, I think this becomes a > discussion about what subset of the semantics of CPython's builtins are > language-specified vs implementation-dependent; my argument is that just > because something results in an observable behavioral difference doesn't > necessarily mean that it's a change in language semantics, if it's just a > change in the implementation-dependent behavior. > > > On Tue, Mar 18, 2014 at 9:54 PM, Stephen J. Turnbull > wrote: >> >> Kevin Modzelewski writes: >> >> > I think in this case, though, if we say for the sake of argument >> > that the guaranteed semantics of a dictionary lookup are zero or >> >> I don't understand the point of that argument. It's simply false that >> semantics are guaranteed, and all of the dunders might be user >> functions. >> >> > more calls to __hash__ plus zero or more calls to __eq__, then two >> > back-to-back dictionary lookups wouldn't have any observable >> > differences from doing only one, unless you start to make >> > assumptions about the behavior of the implementation. >> >> That's false. The inverse is true: you should allow the possibility of >> observable differences, unless you make assumptions about the behavior >> (implying there are none). >> >> > To me there seems to be a bit of a gap between seeing a dictionary >> > lookup and knowing the exact sequence of user-functions that get >> > called, far more than for example something like "a < b". >> >> The point here is that we *know* that there may be a user function >> (the dunder that implements []) being called, and it is very hard to >> determine that that function is pure. >> >> Your example of a caching hash is exactly the kind of impure function >> that one would expect, but who knows what might be called -- there >> could be a reference to a database on Mars involved (do we have a >> vehicle on Mars at the moment? anyway...), which calls a pile of >> Twisted code, and has latencies of many seconds. >> >> So Steven is precisely right -- in order to allow this optimization, >> it would have to be explicitly allowed. >> >> Like Steven, I have no strong feeling against it, but then, I don't >> have a program talking to a deep space vehicle in my near future. >> Darn it! :-( >> > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com > we're discussing builtin dicts ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
Kevin Modzelewski writes: > Sorry, I definitely didn't mean to imply that this kind of > optimization is valid on arbitrary subscript expressions; I thought > we had restricted ourselves to talking about builtin dicts. Ah, maybe so -- Maciej made that clear later for PyPy. My bad. (With the caveat that IIUC Python does not include the technology for detecting for sure that you've got a builtin dict -- a given instance might even be monkey-patched.) > If we do, I think this becomes a discussion about what subset of > the semantics of CPython's builtins are language-specified vs > implementation-dependent; my argument is that just because > something results in an observable behavioral difference doesn't > necessarily mean that it's a change in language semantics, if it's > just a change in the implementation-dependent behavior. I think you're wrong there. Python makes very strong guarantees of backward compatibility; there really isn't that much left to be implementation-dependent once a feature has been introduced and released. Things are a lot more flexible at the introduction of a feature (eg, the decorator example where the specification of the feature clearly involves evaluating an expression twice, but the implementation doesn't). Although I doubt anybody considers that a big deal (and if they did, they'd fix the documentation). But I don't think a *change* in that would be consided "not a change in language semantics." That doesn't mean such a change wouldn't be allowed in a 3.x.0 release. Just that it would be called a change. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'Add/Remove Programs' entry missing for 'current user only' 32-bit installations on 64-bit Windows
Hi. > Did you try the 3.4.0rc3 "repair" installation? That should have > undeleted (repaired) the files that you had manually deleted. Just tested this out and repairing the same matching installation does revert the removed files. It does not reinstall pip and setuptools packages but that can be done easily by hand and so can be filed as a minor enhancement issue for the future. This minor problem reported as issue #20983. http://bugs.python.org/issue20983 Best regards, Jurko Gospodnetić ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'Add/Remove Programs' entry missing for 'current user only' 32-bit installations on 64-bit Windows
Hi. On 19.3.2014. 16:38, "Martin v. Löwis" wrote: Am 17.03.14 22:10, schrieb Jurko Gospodnetić: Fixing this required manually cleaning up leftover CPython 3.4.0rc3 windows installer registry entries. Note that the issue could not be fixed by using the CPython 3.4.0rc3 installer as it failed due to the same problem. Did you try the 3.4.0rc3 "repair" installation? That should have undeleted (repaired) the files that you had manually deleted. I tried different things back then, and I'm 90% sure I tried that one as well. Unfortunately, my brain is already trying to block out that painful afternoon so I can't tell you for certain. :-) But, all kidding aside, I'll try to reproduce the issue again and let you know the exact details. Should I open a 'Add/Remove Programs' dialog related issue in the CPython issue tracker? Please do. It would also good if somebody volunteered to reproduce the issue. Will do. FYI, I did reproduce the 'Add/Remove Programs' dialog issue on another PC in the mean time. Also Windows 7 x64 though. Unfortunately, I don't have other 64-bit OSs readily available at the moment. And possibly a separate one for making CPython installations not fail without possible recovery if a previous CPython installation has already been removed? Please don't. Or, at least, make it more specific. If you have manually corrupted your Python installation (by deleting essential files), you have to accept that the system installation procedures will fail. It might be possible to recover from the loss of a specific such file; for that, we would have to identify what the file is. However, the standard procedure should be to repair the installation before attempting an upgrade. Ok, as I stated above, I'll get back to you on this once I reproduce the 'corruption' issue again. Oh, and many thanks for replying! :-) Best regards, Jurko Gospodnetić ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] asyncio.wait(FIRST_COMPLETED) returns more than one completions - 3.4rc2
Code below has been written with the intension of acquiring ONLY one lock. There are two issues: 1- Sometimes it returns more than one lock in done. 2- Sometimes, even if wait exits with zero or one locks, it seems there are other locks are acquired too. Though, I couldn't isolate the exact case for this. It sounds like some background shield() is at works. I kindly request your comments. locks = [some asyncio.Locks...] sel = [Pack(l.acquire(), l) for l in locks] done, pend = asyncio.wait(sel, timeout=10, return_when=FIRST_COMPLETED) ... @coroutine def Pack(co, obj): yield from co return obj Regards, Imran ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] asyncio.wait(FIRST_COMPLETED) returns more than one completions - 3.4rc2
Hi Imran, The python-dev list is not the place to ask questions about the usage of Python modules or features. However, since you are asking an asyncio-related question, you should be welcome at the python-tulip list: https://groups.google.com/forum/?fromgroups#!forum/python-tulip --Guido On Wed, Mar 19, 2014 at 11:07 AM, Imran Geriskovan < imran.gerisko...@gmail.com> wrote: > Code below has been written with the intension of acquiring ONLY one lock. > There are two issues: > > 1- Sometimes it returns more than one lock in done. > 2- Sometimes, even if wait exits with zero or one locks, it seems > there are other locks are acquired too. Though, I couldn't isolate > the exact case for this. > > It sounds like some background shield() is at works. > > I kindly request your comments. > > > locks = [some asyncio.Locks...] > sel = [Pack(l.acquire(), l) for l in locks] > done, pend = asyncio.wait(sel, timeout=10, return_when=FIRST_COMPLETED) > ... > @coroutine > def Pack(co, obj): > yield from co > return obj > > > Regards, > Imran > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'Add/Remove Programs' entry missing for 'current user only' 32-bit installations on 64-bit Windows
Hi. Should I open a 'Add/Remove Programs' dialog related issue in the CPython issue tracker? Please do. It would also good if somebody volunteered to reproduce the issue. Opened issue #20984. http://bugs.python.org/issue20984 Anyone have Windows 8 x64 available to try this out by any chance? Best regards, Jurko Gospodnetić ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Making proxy types easier to write and maintain
Hello, It is known to be cumbersome to write a proxy type that will correctly proxy all special methods (this has to be done manually on the type, since special methods are not looked up on the instance: a __getattr__ method would not work). Recently we've had reports of two stdlib types that forgot to implement some special methods: - weakref.proxy doesn't implement __reversed__: http://bugs.python.org/issue19359 - mock.MagicMock doesn't implement __truediv__: http://bugs.python.org/issue20968 In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a "proxy protocol" (__proxy__ / tp_proxy) that would be used as a fallback by _PyObject_LookupSpecial to fetch the lookup target, i.e.: def _PyObject_LookupSpecial(obj, name): tp = type(obj) try: return getattr(tp, name) except AttributeError: return getattr(tp.tp_proxy(), name) What do you think? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unit tests for error messages
On Wed, 19 Mar 2014 10:53:31 -0700 Ethan Furman wrote: > I just made a change to some error messages [1] (really, just one): > > old behavior: > >'%o' % 3.14 >'float' object cannot be interpreted as an integer > > new behavior: > >'%o' % 3.14 >%o format: an integer is required, not float > > Would we normally add a test for that? Depends if you're fearing a regression. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] unit tests for error messages
I just made a change to some error messages [1] (really, just one): old behavior: '%o' % 3.14 'float' object cannot be interpreted as an integer new behavior: '%o' % 3.14 %o format: an integer is required, not float Would we normally add a test for that? -- ~Ethan~ [1] Issue 19995: http://bugs.python.org/issue19995 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 'Add/Remove Programs' entry missing for 'current user only' 32-bit installations on 64-bit Windows
Sure, I'll check it this evening. Jurko Gospodnetić wrote on 03/19/2014 01:41:19 PM: > >Hi. > > >>Should I open a 'Add/Remove Programs' dialog related issue in the > >> CPython issue tracker? > > > > Please do. It would also good if somebody volunteered to reproduce > > the issue. > >Opened issue #20984. > >http://bugs.python.org/issue20984 > >Anyone have Windows 8 x64 available to try this out by any chance? > >Best regards, > Jurko Gospodnetić > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > ahettinger%40prominic.net___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unit tests for error messages
Am 19.03.2014 19:55, schrieb Antoine Pitrou: > On Wed, 19 Mar 2014 10:53:31 -0700 > Ethan Furman wrote: > >> I just made a change to some error messages [1] (really, just one): >> >> old behavior: >> >>'%o' % 3.14 >>'float' object cannot be interpreted as an integer >> >> new behavior: >> >>'%o' % 3.14 >>%o format: an integer is required, not float >> >> Would we normally add a test for that? > > Depends if you're fearing a regression. If he fears regressions in error messages, he shouldn't be changing them in the first place :) Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Making proxy types easier to write and maintain
On Wed Mar 19 2014 at 2:46:48 PM, Antoine Pitrou wrote: > > Hello, > > It is known to be cumbersome to write a proxy type that will correctly > proxy all special methods (this has to be done manually on the type, > since special methods are not looked up on the instance: a __getattr__ > method would not work). > > Recently we've had reports of two stdlib types that forgot to > implement some special methods: > - weakref.proxy doesn't implement __reversed__: > http://bugs.python.org/issue19359 > - mock.MagicMock doesn't implement __truediv__: > http://bugs.python.org/issue20968 > > In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a > "proxy > protocol" (__proxy__ / tp_proxy) that would be used as a fallback by > _PyObject_LookupSpecial to fetch the lookup target, i.e.: > > def _PyObject_LookupSpecial(obj, name): > tp = type(obj) > try: > return getattr(tp, name) > except AttributeError: > return getattr(tp.tp_proxy(), name) > > What do you think? > Without having the code in front of me, would this only be for magic methods and attributes, or all attributes? IOW would this mean that if I assign an object to __proxy__ it would take care of the uses of __getattr__ for proxying? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 19 March 2014 18:46, Antoine Pitrou wrote: > In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a > "proxy > protocol" (__proxy__ / tp_proxy) that would be used as a fallback by > _PyObject_LookupSpecial to fetch the lookup target, i.e.: > > def _PyObject_LookupSpecial(obj, name): > tp = type(obj) > try: > return getattr(tp, name) > except AttributeError: > return getattr(tp.tp_proxy(), name) > > What do you think? Would that increase the size of type objects? Would that matter? (There's a similar question that came up in the thread about adding the @ operator over on python-ideas, which is what made me think of it...) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Wed, 19 Mar 2014 19:32:50 + Brett Cannon wrote: > > > > In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a > > "proxy > > protocol" (__proxy__ / tp_proxy) that would be used as a fallback by > > _PyObject_LookupSpecial to fetch the lookup target, i.e.: > > > > def _PyObject_LookupSpecial(obj, name): > > tp = type(obj) > > try: > > return getattr(tp, name) > > except AttributeError: > > return getattr(tp.tp_proxy(), name) > > > > What do you think? > > > > Without having the code in front of me, would this only be for magic > methods and attributes, or all attributes? IOW would this mean that if I > assign an object to __proxy__ it would take care of the uses of __getattr__ > for proxying? In a first approach it would be only for magic methods and attributes. It if proves successful, perhaps it can be later expanded to also be used for regular lookups. By the way, Benjamin pointed me to a prior discussion: http://bugs.python.org/issue643841 Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Wed, 19 Mar 2014 20:15:15 + Paul Moore wrote: > On 19 March 2014 18:46, Antoine Pitrou wrote: > > In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a > > "proxy > > protocol" (__proxy__ / tp_proxy) that would be used as a fallback by > > _PyObject_LookupSpecial to fetch the lookup target, i.e.: > > > > def _PyObject_LookupSpecial(obj, name): > > tp = type(obj) > > try: > > return getattr(tp, name) > > except AttributeError: > > return getattr(tp.tp_proxy(), name) > > > > What do you think? > > Would that increase the size of type objects? Would that matter? > (There's a similar question that came up in the thread about adding > the @ operator over on python-ideas, which is what made me think of > it...) One additional slot is one additional pointer field, which is mostly trivial in a type object. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On 20 Mar 2014 02:37, "Stephen J. Turnbull" wrote: > > Kevin Modzelewski writes: > > > Sorry, I definitely didn't mean to imply that this kind of > > optimization is valid on arbitrary subscript expressions; I thought > > we had restricted ourselves to talking about builtin dicts. > > Ah, maybe so -- Maciej made that clear later for PyPy. My bad. (With > the caveat that IIUC Python does not include the technology for > detecting for sure that you've got a builtin dict -- a given instance > might even be monkey-patched.) CPython doesn't, PyPy does. There are many reasons Armin Rigo stopped working on psyco in CPython and created PyPy instead - one of them is that making optimisations like this maintainable for a dynamic language like Python essentially required inventing a whole new approach to creating dynamic language interpreters. > > If we do, I think this becomes a discussion about what subset of > > the semantics of CPython's builtins are language-specified vs > > implementation-dependent; my argument is that just because > > something results in an observable behavioral difference doesn't > > necessarily mean that it's a change in language semantics, if it's > > just a change in the implementation-dependent behavior. > > I think you're wrong there. Python makes very strong guarantees of > backward compatibility; there really isn't that much left to be > implementation-dependent once a feature has been introduced and > released. Correct, but I think this discussion has established that "how many times dict lookup calls __eq__ on the key" is one such thing. In CPython, it already varies based on: - dict contents (due to the identity check and the distribution of entries across hash buckets) - pointer size (due to the hash bucket distribution differing between 32 bit and 64 bit builds) - dict tuning parameters (there are some settings in the dict implementation that affect when dicts resize up and down, etc, which can mean the hash bucket distribution may already change without much notice in feature releases) So that part of this PyPy optimisation shouldn't be controversial, leaving the matter of only calling __hash__ on the lookup key once rather than twice. Since "hash(x)" changing with time is just a straight up bug in the implementation of "x", that part also sounds fine. So yeah, it's certainly a subtle point, but I agree with Maciej that the PyPy team have found a legitimate optimisation opportunity here. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On 20 Mar 2014 07:38, "Nick Coghlan" wrote: > > Correct, but I think this discussion has established that "how many times dict lookup calls __eq__ on the key" is one such thing. In CPython, it already varies based on: > > - dict contents (due to the identity check and the distribution of entries across hash buckets) > - pointer size (due to the hash bucket distribution differing between 32 bit and 64 bit builds) > - dict tuning parameters (there are some settings in the dict implementation that affect when dicts resize up and down, etc, which can mean the hash bucket distribution may already change without much notice in feature releases) I just realised that hash randomisation also comes into play here - the distribution of entries across hash buckets is inherently variable between runs for any key types that rely directly or indirectly on a randomised hash. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 20 Mar 2014 06:24, "Antoine Pitrou" wrote: > > On Wed, 19 Mar 2014 19:32:50 + > Brett Cannon wrote: > > > > > > In http://bugs.python.org/issue19359#msg213530 I proposed to introduce a > > > "proxy > > > protocol" (__proxy__ / tp_proxy) that would be used as a fallback by > > > _PyObject_LookupSpecial to fetch the lookup target, i.e.: > > > > > > def _PyObject_LookupSpecial(obj, name): > > > tp = type(obj) > > > try: > > > return getattr(tp, name) > > > except AttributeError: > > > return getattr(tp.tp_proxy(), name) > > > > > > What do you think? > > > > > > > Without having the code in front of me, would this only be for magic > > methods and attributes, or all attributes? IOW would this mean that if I > > assign an object to __proxy__ it would take care of the uses of __getattr__ > > for proxying? > > In a first approach it would be only for magic methods and attributes. > It if proves successful, perhaps it can be later expanded to also be > used for regular lookups. > > By the way, Benjamin pointed me to a prior discussion: > http://bugs.python.org/issue643841 Graeme Dumpleton has also subsequently written a library to handle easier creation of correct proxy types: https://pypi.python.org/pypi/wrapt It isn't as simple as changing one lookup function though - abstract.c reads the type slots directly, and those are already hairy performance critical code paths (especially the binary operator type dispatch). Cheers, Nick. > > Regards > > Antoine. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Thu, 20 Mar 2014 07:54:39 +1000 Nick Coghlan wrote: > > Graeme Dumpleton has also subsequently written a library to handle easier > creation of correct proxy types: https://pypi.python.org/pypi/wrapt (is "Graeme" another spelling for "Graham"?) > It isn't as simple as changing one lookup function though - abstract.c > reads the type slots directly, and those are already hairy performance > critical code paths (especially the binary operator type dispatch). Oh indeed, you're right. I think my proposal is taking water. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] unittest assertRaisesRegex bug?
Here's the code in question:
class PsuedoFloat:
def __init__(self, value):
self.value = float(value)
def __int__(self):
return int(self.value)
pi = PsuedoFloat(3.1415)
self.assertRaisesRegex(TypeError, '%x format: an integer is required,
not PsuedoFloat', '%x'.__mod__, pi),
Here's the exception:
==
ERROR: test_formatting (test.test_unicode.UnicodeTest)
--
TypeError: 'PsuedoFloat' object is not callable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", line
1156, in test_formatting
self.assertRaisesRegex(TypeError, '%c'.__mod__, pi),
File "/home/ethan/source/python/issue19995/Lib/unittest/case.py", line 1235,
in assertRaisesRegex
return context.handle('assertRaisesRegex', callable_obj, args, kwargs)
File "/home/ethan/source/python/issue19995/Lib/unittest/case.py", line 161,
in handle
callable_obj(*args, **kwargs)
File "/home/ethan/source/python/issue19995/Lib/unittest/case.py", line 190,
in __exit__
if not expected_regex.search(str(exc_value)):
AttributeError: 'method-wrapper' object has no attribute 'search'
--
At worst, I was expecting a difference in the TypeError exception message; I have no idea why `pi` is being called.
From the docs:
http://docs.python.org/3/library/unittest.html#unittest.TestCase.assertRaisesRegex
assertRaisesRegex(exception, regex, callable, *args, **kwds)
Like assertRaises() but also tests that regex matches on the string
representation
of the raised exception. regex may be a regular expression object or a
string
containing a regular expression suitable for use by re.search(). Examples:
self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$", int, 'XYZ')
Am I correct in thinking this is a bug?
--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest assertRaisesRegex bug?
On Wed, 19 Mar 2014 14:37:42 -0700 Ethan Furman wrote: > Here's the code in question: > > class PsuedoFloat: > def __init__(self, value): > self.value = float(value) > def __int__(self): > return int(self.value) > > pi = PsuedoFloat(3.1415) > > self.assertRaisesRegex(TypeError, '%x format: an integer is > required, not PsuedoFloat', '%x'.__mod__, pi), > > Here's the exception: > > == > ERROR: test_formatting (test.test_unicode.UnicodeTest) > -- > TypeError: 'PsuedoFloat' object is not callable > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): >File "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", line > 1156, in test_formatting > self.assertRaisesRegex(TypeError, '%c'.__mod__, pi), This is certainly not the code you are showing above. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest assertRaisesRegex bug?
On 03/19/2014 03:13 PM, Antoine Pitrou wrote: On Wed, 19 Mar 2014 14:37:42 -0700 Ethan Furman wrote: Here's the code in question: class PsuedoFloat: def __init__(self, value): self.value = float(value) def __int__(self): return int(self.value) pi = PsuedoFloat(3.1415) self.assertRaisesRegex(TypeError, '%x format: an integer is required, not PsuedoFloat', '%x'.__mod__, pi), Here's the exception: == ERROR: test_formatting (test.test_unicode.UnicodeTest) -- TypeError: 'PsuedoFloat' object is not callable During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", line 1156, in test_formatting self.assertRaisesRegex(TypeError, '%c'.__mod__, pi), This is certainly not the code you are showing above. More words, please! :) Do you mean you agree it's a bug, or do you mean you think I misstepped in reporting what's going on? -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest assertRaisesRegex bug?
On Wed, 19 Mar 2014 15:17:53 -0700 Ethan Furman wrote: > On 03/19/2014 03:13 PM, Antoine Pitrou wrote: > > On Wed, 19 Mar 2014 14:37:42 -0700 > > Ethan Furman wrote: > >> Here's the code in question: > >> > >> class PsuedoFloat: > >> def __init__(self, value): > >> self.value = float(value) > >> def __int__(self): > >> return int(self.value) > >> > >> pi = PsuedoFloat(3.1415) > >> > >> self.assertRaisesRegex(TypeError, '%x format: an integer is > >> required, not PsuedoFloat', '%x'.__mod__, pi), > >> > >> Here's the exception: > >> > >> == > >> ERROR: test_formatting (test.test_unicode.UnicodeTest) > >> -- > >> TypeError: 'PsuedoFloat' object is not callable > >> > >> During handling of the above exception, another exception occurred: > >> > >> Traceback (most recent call last): > >> File "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", > >> line 1156, in test_formatting > >> self.assertRaisesRegex(TypeError, '%c'.__mod__, pi), > > > > This is certainly not the code you are showing above. > > More words, please! :) I mean the line shown just above in the traceback does not match the code you presented at the top, and that line clearly has a missing regex pattern. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest assertRaisesRegex bug?
On 03/19/2014 03:57 PM, Antoine Pitrou wrote: On Wed, 19 Mar 2014 15:17:53 -0700 Ethan Furman wrote: On 03/19/2014 03:13 PM, Antoine Pitrou wrote: On Wed, 19 Mar 2014 14:37:42 -0700 Ethan Furman wrote: Here's the code in question: class PsuedoFloat: def __init__(self, value): self.value = float(value) def __int__(self): return int(self.value) pi = PsuedoFloat(3.1415) self.assertRaisesRegex(TypeError, '%x format: an integer is required, not PsuedoFloat', '%x'.__mod__, pi), Here's the exception: == ERROR: test_formatting (test.test_unicode.UnicodeTest) -- TypeError: 'PsuedoFloat' object is not callable During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", line 1156, in test_formatting self.assertRaisesRegex(TypeError, '%c'.__mod__, pi), This is certainly not the code you are showing above. More words, please! :) I mean the line shown just above in the traceback does not match the code you presented at the top, and that line clearly has a missing regex pattern. A regex pattern can be a literal, yes? In which case exception -> TypeError regex -> '%x format: an integer is required, not PsuedoFloat' callable -> '%x'.__mod__ *args -> pi **kwargs -> None So, unless you can point to where I've gone wrong with the above (which is why I posted in the first place), I think we have a bug in unittest. Also: self.assertRaisesRegex(TypeError, '%x format: an integer is required, not float','%x'.__mod__, 3.14), self.assertRaisesRegex(TypeError, '%X format: an integer is required, not float','%X'.__mod__, 2.11), self.assertRaisesRegex(TypeError, '%o format: an integer is required, not float','%o'.__mod__, 1.79), these lines all work just fine. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest assertRaisesRegex bug?
On Wed, Mar 19, 2014 at 4:13 PM, Ethan Furman wrote: > On 03/19/2014 03:57 PM, Antoine Pitrou wrote: > >> On Wed, 19 Mar 2014 15:17:53 -0700 >> Ethan Furman wrote: >> >>> On 03/19/2014 03:13 PM, Antoine Pitrou wrote: >>> On Wed, 19 Mar 2014 14:37:42 -0700 Ethan Furman wrote: > Here's the code in question: > >class PsuedoFloat: >def __init__(self, value): >self.value = float(value) >def __int__(self): >return int(self.value) > >pi = PsuedoFloat(3.1415) > >self.assertRaisesRegex(TypeError, '%x format: an integer > is required, not PsuedoFloat', '%x'.__mod__, pi), > > Here's the exception: > > == > ERROR: test_formatting (test.test_unicode.UnicodeTest) > -- > TypeError: 'PsuedoFloat' object is not callable > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", > line 1156, in test_formatting >self.assertRaisesRegex(TypeError, '%c'.__mod__, pi), > This is certainly not the code you are showing above. >>> >>> More words, please! :) >>> >> >> I mean the line shown just above in the traceback does not match the >> code you presented at the top, and that line clearly has a missing >> regex pattern. >> > > A regex pattern can be a literal, yes? In which case > > exception -> TypeError > regex -> '%x format: an integer is required, not PsuedoFloat' > callable -> '%x'.__mod__ > *args -> pi > **kwargs -> None > > So, unless you can point to where I've gone wrong with the above (which is > why I posted in the first place), I think we have a bug in unittest. > > Also: > > self.assertRaisesRegex(TypeError, '%x format: an integer is > required, not float','%x'.__mod__, 3.14), > self.assertRaisesRegex(TypeError, '%X format: an integer is > required, not float','%X'.__mod__, 2.11), > self.assertRaisesRegex(TypeError, '%o format: an integer is > required, not float','%o'.__mod__, 1.79), > > these lines all work just fine. > What Antoine is trying to tell you is that the traceback you pasted shows this: File "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", line 1156, in test_formatting self.assertRaisesRegex(TypeError, '%c'.__mod__, pi), ... which is passing '%c'.__mod__ as the 'regex' argument. '%c'.__mod__ is a method of a builtin type, a 'method-wrapper' object, which is why you get the error you're getting: AttributeError: 'method-wrapper' object has no attribute 'search'. -- Thomas Wouters Hi! I'm an email virus! Think twice before sending your email to help me spread! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unit tests for error messages
On Wed, 19 Mar 2014 20:32:38 +0100, Georg Brandl wrote:
> Am 19.03.2014 19:55, schrieb Antoine Pitrou:
> > On Wed, 19 Mar 2014 10:53:31 -0700
> > Ethan Furman wrote:
> >
> >> I just made a change to some error messages [1] (really, just one):
> >>
> >> old behavior:
> >>
> >>'%o' % 3.14
> >>'float' object cannot be interpreted as an integer
> >>
> >> new behavior:
> >>
> >>'%o' % 3.14
> >>%o format: an integer is required, not float
> >>
> >> Would we normally add a test for that?
> >
> > Depends if you're fearing a regression.
>
> If he fears regressions in error messages, he shouldn't be changing them
> in the first place :)
In this type of situation I will often write a unit test that makes
sure that the piece of information I just added to the message is really
in the message ('%o', in this case), using assertRaisesRegex. I don't
think it is required, but I like to do it, because it would indeed be
a regression if that information stopped appearing in the message.
--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On Wed, Mar 19, 2014 at 11:01:39PM +0100, Antoine Pitrou wrote: > On Thu, 20 Mar 2014 07:54:39 +1000 > Nick Coghlan wrote: > > > > Graeme Dumpleton has also subsequently written a library to handle easier > > creation of correct proxy types: https://pypi.python.org/pypi/wrapt > > (is "Graeme" another spelling for "Graham"?) Yes. > > It isn't as simple as changing one lookup function though - abstract.c > > reads the type slots directly, and those are already hairy performance > > critical code paths (especially the binary operator type dispatch). > > Oh indeed, you're right. I think my proposal is taking water. Nevertheless, I would like to see a standard way to proxy dunder methods. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest assertRaisesRegex bug?
On Wed, 19 Mar 2014 16:41:10 -0700, Thomas Wouters wrote: > On Wed, Mar 19, 2014 at 4:13 PM, Ethan Furman wrote: > > > On 03/19/2014 03:57 PM, Antoine Pitrou wrote: > > > >> On Wed, 19 Mar 2014 15:17:53 -0700 > >> Ethan Furman wrote: > >> > >>> On 03/19/2014 03:13 PM, Antoine Pitrou wrote: > >>> > On Wed, 19 Mar 2014 14:37:42 -0700 > Ethan Furman wrote: > > During handling of the above exception, another exception occurred: > > > > Traceback (most recent call last): > > File > > "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", > > line 1156, in test_formatting > >self.assertRaisesRegex(TypeError, '%c'.__mod__, pi), > > > > This is certainly not the code you are showing above. > > >>> > >>> More words, please! :) > >>> > >> > >> I mean the line shown just above in the traceback does not match the > >> code you presented at the top, and that line clearly has a missing > >> regex pattern. > >> > > > > A regex pattern can be a literal, yes? In which case > > > > exception -> TypeError > > regex -> '%x format: an integer is required, not PsuedoFloat' > > callable -> '%x'.__mod__ > > *args -> pi > > **kwargs -> None > > > > So, unless you can point to where I've gone wrong with the above (which is > > why I posted in the first place), I think we have a bug in unittest. > > > > Also: > > > > self.assertRaisesRegex(TypeError, '%x format: an integer is > > required, not float','%x'.__mod__, 3.14), > > self.assertRaisesRegex(TypeError, '%X format: an integer is > > required, not float','%X'.__mod__, 2.11), > > self.assertRaisesRegex(TypeError, '%o format: an integer is > > required, not float','%o'.__mod__, 1.79), > > > > these lines all work just fine. > > > > What Antoine is trying to tell you is that the traceback you pasted shows > this: > > File "/home/ethan/source/python/issue19995/Lib/test/test_unicode.py", > line 1156, in test_formatting > self.assertRaisesRegex(TypeError, '%c'.__mod__, pi), > > ... which is passing '%c'.__mod__ as the 'regex' argument. '%c'.__mod__ is > a method of a builtin type, a 'method-wrapper' object, which is why you get > the error you're getting: AttributeError: 'method-wrapper' object has no > attribute 'search'. I think http://bugs.python.org/issue20145 is relevant here. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest assertRaisesRegex bug?
On 03/19/2014 04:41 PM, Thomas Wouters wrote: What Antoine is trying to tell you is that the traceback you pasted shows this: File "/home/ethan/source/python/__issue19995/Lib/test/test___unicode.py", line 1156, in test_formatting self.assertRaisesRegex(__TypeError, '%c'.__mod__, pi), ... which is passing '%c'.__mod__ as the 'regex' argument. Argh. Thank you, Thomas. Thank you, and my apologies for being so dense, Antoine. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making proxy types easier to write and maintain
On 20 Mar 2014 10:34, "Steven D'Aprano" wrote: > > On Wed, Mar 19, 2014 at 11:01:39PM +0100, Antoine Pitrou wrote: > > On Thu, 20 Mar 2014 07:54:39 +1000 > > Nick Coghlan wrote: > > > > > > Graeme Dumpleton has also subsequently written a library to handle easier > > > creation of correct proxy types: https://pypi.python.org/pypi/wrapt > > > > (is "Graeme" another spelling for "Graham"?) > > Yes. > > > > > It isn't as simple as changing one lookup function though - abstract.c > > > reads the type slots directly, and those are already hairy performance > > > critical code paths (especially the binary operator type dispatch). > > > > Oh indeed, you're right. I think my proposal is taking water. > > Nevertheless, I would like to see a standard way to proxy dunder > methods. I had vague plans to bug Graham (yay, spelling!) about proposing wrapt for inclusion in 3.5 (possibly split between functools and types rather than as a completely new module, though). functools.wraps was always intended as a near term workaround that, like most sufficiently effective workarounds, ended lasting years before its limitations irritated anyone enough for them to go back and do it right :) Cheers, Nick. > > > -- > Steven > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
