from:"Stefan Richthofer"

Re: [Python-Dev] sum(...) limitation

2014-08-12 Thread Stefan Richthofer

I know, I have nothing to decide here, since I'm no contributer and just a silent watcher on this list.

However I just wanted to point out I fully agree with Chris Barker's position. Couldn't have stated

it better. Performance should be interpreter implementation issue, not language issue.

> 2) add a special case for strings that is fast and efficient -- may be as simple as calling "".join() under the hood --no more code than the exception check.

I would give it a +1 if my opinion counts anything.

Cheers

Stefan

Gesendet: Dienstag, 12. August 2014 um 21:11 Uhr
Von: "Chris Barker"
An: Kein Empfänger
Cc: "Python Dev"
Betreff: Re: [Python-Dev] sum(...) limitation

On Mon, Aug 11, 2014 at 11:07 PM, Stephen J. Turnbull wrote:

I'm referring to removing the unnecessary information that there's a
better way to do it, and simply raising an error (as in Python 3.2,
say) which is all a RealProgrammer[tm] should ever need!

I can't imagine anyone is suggesting that -- disallow it, but don't tell anyone why?

The only thing that is remotely on the table here is:

1) remove the special case for strings -- buyer beware -- but consistent and less "ugly"

2) add a special case for strings that is fast and efficient -- may be as simple as calling "".join() under the hood --no more code than the exception check.

And I doubt anyone really is pushing for anything but (2)

Steven Turnbull wrote:

IMO we'd also want a homogeneous_iterable ABC

Actually, I've thought for years that that would open the door to a lot of optimizations -- but that's a much broader question that sum(). I even brought it up probably over ten years ago -- but no one was the least bit iinterested -- nor are they now -- I now this was a rhetorical suggestion to make the point about what not to do

Because obviously we'd want the
attractive nuisance of "if you have __add__, there's a default
definition of __sum__"

now I'm confused -- isn't that exactly what we have now?

It's possible that Python could provide some kind of feature that
would allow an optimized sum function for every type that has __add__,
but I think this will take a lot of thinking.

does it need to be every type? As it is the common ones work fine already except for strings -- so if we add an optimized string sum() then we're done.

*Somebody* will do it
(I don't think anybody is +1 on restricting sum() to a subset of types
with __add__).

uhm, that's exactly what we have now -- you can use sum() with anything that has an __add__, except strings. Ns by that logic, if we thought there were other inefficient use cases, we'd restrict those too.

But users can always define their own classes that have a __sum__ and are really inefficient -- so unless sum() becomes just for a certain subset of built-in types -- does anyone want that? Then we are back to the current situation:

sum() can be used for any type that has an __add__ defined.

But naive users are likely to try it with strings, and that's bad, so we want to prevent that, and have a special case check for strings.

What I fail to see is why it's better to raise an exception and point users to a better way, than to simply provide an optimization so that it's a mute issue.

The only justification offered here is that will teach people that summing strings (and some other objects?) is order(N^2) and a bad idea. But:

a) Python's primary purpose is practical, not pedagogical (not that it isn't great for that)

b) I doubt any naive users learn anything other than "I can't use sum() for strings, I should use "".join()". Will they make the leap to "I shouldn't use string concatenation in a loop, either"? Oh, wait, you can use string concatenation in a loop -- that's been optimized. So will they learn: "some types of object shave poor performance with repeated concatenation and shouldn't be used with sum(). So If I write such a class, and want to sum them up, I'll need to write an optimized version of that code"?

I submit that no naive user is going to get any closer to a proper understanding of algorithmic Order behavior from this small hint. Which leaves no reason to prefer an Exception to an optimization.

One other point: perhaps this will lead a naive user into thinking -- "sum() raises an exception if I try to use it inefficiently, so it must be OK to use for anything that doesn't raise an exception" -- that would be a bad lesson to mis-learn

-Chris

PS:

Armin Rigo wrote:

It also improves a
lot the precision of sum(list_of_floats) (though not reaching the same
precision levels of math.fsum()).

while we are at it, having the default sum() for floats be fsum() would be nice -- I'd rather the default was better accuracy loser performance. Folks that really care about performance could call math.fastsum(), or really, use numpy...

This does turn sum() into a function that does type-based di

[Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

2014-10-25 Thread Stefan Richthofer

Hello developers,

I observed strange behaviour in CPython (tested in 2.7.5 and 3.3.3)
regarding object resurrection.
Yes, resurrection is evil, but it is a valid scenario. If an object is
resurrected via its finalizer __del__, sometimes its unique id value as
returned from id() changes. Additionally the list of weak references
pointing to it as returned by weakref.getweakrefs(...) breaks (i.e. is
suddenly empty, assuming it wasn't before).
These issues only arise if the resurrection occurs during
cyclic garbage collection. If the object is finalized because its refcount
drops to zero, everything is fine. See the attached test-file.

Is this behaviour intended or is it a bug? If so, which variant is the bug
and which is right? I can hardly believe that whether id() is preserved
should depend on whether the garbage was cyclic or not.

This might appear of low relevance to you, since no sane program intentionally
performs resurrection. However I originally became aware of the issue
in Jython (where it not only occurs for cyclic garbage but in every
resurrection-case), c.f. http://bugs.jython.org/issue2224.
I am interested in this because I am implementing native gc support
in JyNI and need to understand these details to do it right.


Thanks in advance!

Stefanimport unittest
import gc
import time
import weakref

class ReferentDummy:
def __init__(self, name):
self.name = name

def __str__(self):
return self.name

class ResurrectionDummy:
def __del__(self):
ResurrectionDummy.resurrected = self.toResurrect

class GCDetector():
gcIndex = 0

def __del__(self):
GCDetector.gcIndex += 1

maxGCRun = 10

def runGC():
"""
This is needed for Jython, since theoretically Java gc is not guaranteed to
run if gc.collect is called; the run is only attempted. This method assures
that actually a gc run happens.
"""
currentIndex = GCDetector.gcIndex
gcCount = 0
detector = GCDetector()
detector = None
while currentIndex == GCDetector.gcIndex and gcCount < maxGCRun:
gc.collect()
gcCount += 1
time.sleep(0.1)

class GCTests(unittest.TestCase):
def test_id_after_resurrection(self):
l = ["ab"]
rd = ResurrectionDummy()
rd.toResurrect = l
savedId = id(l)
l = None
rd = None
runGC() #needed for Jython etc, even though no cyclic trash appears
self.assertEqual(id(ResurrectionDummy.resurrected), savedId)

def test_id_after_resurrection_cyclic(self):
#CPython 2.7.5 fails this test
rd = ResurrectionDummy()
l = ["ab", rd]
rd.toResurrect = l
savedId = id(l)
l = None
rd = None
runGC()
self.assertEqual(id(ResurrectionDummy.resurrected), savedId)

def test_weakref_after_resurrection(self):
l = ReferentDummy("ab")
rd = ResurrectionDummy()
rd.toResurrect = l
wref = weakref.ref(l)
self.assertIn(wref, weakref.getweakrefs(l))
l = None
rd = None
runGC() #needed for Jython etc, even though no cyclic trash appears
self.assertIn(wref, weakref.getweakrefs(ResurrectionDummy.resurrected))

def test_weakref_after_resurrection_cyclic(self):
#CPython 2.7.5 fails this test
l = ReferentDummy("ab")
rd = ResurrectionDummy()
rd.toResurrect = l
l.cycleLink = rd
wref = weakref.ref(l)
self.assertIn(wref, weakref.getweakrefs(l))
l = None
rd = None
runGC()
self.assertIn(wref, weakref.getweakrefs(ResurrectionDummy.resurrected))


def test_main():
unittest.main()

if __name__ == "__main__":
test_main()
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

2014-10-25 Thread Stefan Richthofer

Okay, sorry, I was thinking too Jython-like. I fixed runGC() just to
see now that it does not even trigger resurrection, since under
CPython there are no finalizers executed in ref cycles (i.e. I find my
objects in gc.garbage).
So I realize, my xy_cyclic tests are pointless anyway since in cyclic
gc no resurrection can happen.

> The second problem (with weakref) is different: weakrefs are cleared
> before __del__ is called, so resurrection doesn't affect the whole
> process.
It appears weakrefs are only cleared if this is done by gc (where no
resurrection can happen anyway). If a resurrection-performing-__del__ is
just called by ref-count-drop-to-0, weakrefs persist - a behavior that is
very difficult and inefficient to emulate in Jython, but I'll give it
some more thoughts...

However thanks for the help!

-Stefan


> Gesendet: Sonntag, 26. Oktober 2014 um 01:22 Uhr
> Von: "Antoine Pitrou" 
> An: python-dev@python.org
> Betreff: Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes 
> break on object resurrection
>
> 
> Hello Stefan,
> 
> On Sun, 26 Oct 2014 00:20:47 +0200
> "Stefan Richthofer"  wrote:
> > Hello developers,
> > 
> > I observed strange behaviour in CPython (tested in 2.7.5 and 3.3.3)
> > regarding object resurrection.
> 
> Your runGC() function is buggy, it does not run the GC under CPython.
> Fix it and the first problem (with id()) disappears.
> 
> The second problem (with weakref) is different: weakrefs are cleared
> before __del__ is called, so resurrection doesn't affect the whole
> process. Add a callback to the weakref and you'll see it is getting
> called.
> 
> In other words, CPython behaves as expected. Your concern is
> appreciated, though.
> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
> 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

2014-10-26 Thread Stefan Richthofer

>You shouldn't have to emulate that. The exact behavior of GC is allowed to vary between systems.

Yes, of course. I am looking into this for JyNI, which in contrast should emulate CPython behavior as good as possible.

And for such details, -one by one- I am currently weighting up whether it's easier to support it already in Jython or in the layer.

And for aspects where it is feasible, I see nothing wrong to get as close as possible to the reference implementation (and the persistence of weakrefs on resurrection seems to be none of these indeed).

Gesendet: Sonntag, 26. Oktober 2014 um 06:53 Uhr
Von: "Guido van Rossum" 
An: "Stefan Richthofer" 
Cc: "python-dev@python.org" 
Betreff: Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

On Saturday, October 25, 2014, Stefan Richthofer <stefan.richtho...@gmx.de> wrote:

Okay, sorry, I was thinking too Jython-like. I fixed runGC() just to
see now that it does not even trigger resurrection, since under
CPython there are no finalizers executed in ref cycles (i.e. I find my
objects in gc.garbage).
So I realize, my xy_cyclic tests are pointless anyway since in cyclic
gc no resurrection can happen.

> The second problem (with weakref) is different: weakrefs are cleared
> before __del__ is called, so resurrection doesn't affect the whole
> process.
It appears weakrefs are only cleared if this is done by gc (where no
resurrection can happen anyway). If a resurrection-performing-__del__ is
just called by ref-count-drop-to-0, weakrefs persist - a behavior that is
very difficult and inefficient to emulate in Jython, but I'll give it
some more thoughts...

You shouldn't have to emulate that. The exact behavior of GC is allowed to vary between systems.

However thanks for the help!

-Stefan

> Gesendet: Sonntag, 26. Oktober 2014 um 01:22 Uhr
> Von: "Antoine Pitrou" <solip...@pitrou.net>
> An: python-dev@python.org
> Betreff: Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection
>
>
> Hello Stefan,
>
> On Sun, 26 Oct 2014 00:20:47 +0200
> "Stefan Richthofer" <stefan.richtho...@gmx.de> wrote:
> > Hello developers,
> >
> > I observed strange behaviour in CPython (tested in 2.7.5 and 3.3.3)
> > regarding object resurrection.
>
> Your runGC() function is buggy, it does not run the GC under CPython.
> Fix it and the first problem (with id()) disappears.
>
> The second problem (with weakref) is different: weakrefs are cleared
> before __del__ is called, so resurrection doesn't affect the whole
> process. Add a callback to the weakref and you'll see it is getting
> called.
>
> In other words, CPython behaves as expected. Your concern is
> appreciated, though.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org

--
--Guido van Rossum (on iPad)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

2014-10-27 Thread Stefan Richthofer


Your test program performs no resurrection of x.

Interestingly, it does not change behavior if you write

class X(object):
def __del__(self):
X.x = self
print ref()

(Thanks for making me aware of this! My test-case was already
initially the more complex one given below)

But if the resurrection occurs indirectly, the weakref persists:
(I refined it to old-style class, because Jython will support new-style
class finalizers only from 2.7. beta 4 onwards, i.e. the test would be
pointless with any current release)

import weakref, time, gc
class ReferentDummy():
pass

class X():
def __del__(self):
X.y = self.z
print "__del__: "+str(ref())

x = X()
x2 = ReferentDummy()
ref = weakref.ref(x2)
x.z = x2
del x2
del x #Everything is now deleted, isn't it?
gc.collect() #needed in Jython-case
time.sleep(0.2) #wait for Java's async gc to finnish
print ref()
print weakref.getweakrefs(X.y)


---CPython output:
__del__: <__main__.ReferentDummy instance at 0x7fd2603e1950>
<__main__.ReferentDummy instance at 0x7fd2603e1950>
[]

---Jython 2.7 beta 3 output:
__del__: None
None
[]

One can surely argue x2 has never been dead, or see it as "it was killed 
along with x and
then resurrected by x". Jython clearly takes the second point of view 
and also clears weakrefs
to x.z, while CPython does not. Yes, these details probably hardly 
matter in practice (however
could cause subtle bugs when porting complex stuff from CPython to 
Jython), but since I try to

bridge it, I have to look into this more carefully.

Best,

Stefan



On 10/26/2014 06:44 PM, Armin Rigo wrote:

Hi Stefan,

On 26 October 2014 02:50, Stefan Richthofer  wrote:

It appears weakrefs are only cleared if this is done by gc (where no
resurrection can happen anyway). If a resurrection-performing-__del__ is
just called by ref-count-drop-to-0, weakrefs persist -

How do you reach this conclusion?  The following test program seems to
show the opposite, by printing None on Python 2.7.6:

 import weakref
class X(object):
 def __del__(self):
 print ref()
 x = X()
 ref = weakref.ref(x)
 del x


A bientôt,

Armin.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

2014-10-27 Thread Stefan Richthofer

>It's not that resurrection occurs indirectly, it's that the object
>pointed to by "x2" always remains alive

Yes, this is right for CPython, more precisely this is about the definition
of the word "resurrection" (in language-independent sense),
which seems not to be unique.

I already pointed out
"One can surely argue x2 has never been dead, or see it as
"it was killed along with x and then resurrected by x"."

In Java and thus in Jython, it is treated as the second one. An equal
program written in Java or Jython would even call the finalizer of x2 (if
it had one) and clear weakrefs before it is available "again" as a class
attribute of X.
So there actually *is* a notion to refer to this scenario as resurrection.
I admit it is arguable and maybe misleading in CPython case and I was
not aware of the whole behavior when I called the topic "resurrection".

What would still be interesting (at least when Jython 3 is born),
is which of the mentioned behaviors occurs if it is
performed by CPython's cyclic gc (consistently the first one I would guess).
As you pointed out, this is only relevant from 3.4 on since in 2.x etc 
it does
not call finalizers in cycles. (Since I mainly work on Jython or Python 
2.7 I
currently have no 3.4 installed to test this instantaneously. I will 
test it someday...)

Best,

Stefan

On 10/27/2014 03:14 PM, Antoine Pitrou wrote:

On Mon, 27 Oct 2014 14:36:31 +0100
Stefan Richthofer  wrote:

Your test program performs no resurrection of x.

Interestingly, it does not change behavior if you write

class X(object):
  def __del__(self):
  X.x = self
  print ref()

(Thanks for making me aware of this! My test-case was already
initially the more complex one given below)

But if the resurrection occurs indirectly, the weakref persists:

It's not that resurrection occurs indirectly, it's that the object
pointed to by "x2" always remains alive (first as an instance attribute
of x, second as a class attribute of X *before x is deleted*).

Regards

Antoine.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

2014-10-27 Thread Stefan Richthofer




You mean Jython deletes instance attributes before calling __del__ ?


No. I think the term of "object resurrection" usually does not mean bringing
back a deleted object in the sense that memory was already freed.
I think it rather means that nothing referred to an object, so it was on the
"kill-list" of gc or zero-ref-count macro.
During its finalizer call, the object managed to get some reference
again. GC or zero-ref-count macro checks refcount again after the
finalizer call (doesn't it?) and then refrains from the originally triggered
deletion-task.

Where things get weired is how to treat objects (e.g. x2) that are
reachable via the original
object (e.g. x) only.

x becomes unreachable => x2 is unreachable too

CPython behavior:
free x's weakrefs, call x.__del__ => x2 is reachable again => free
memory of x; don't touch x2 at all

Java/Jython behavior:
free all weakrefs, call all finalizers of unreachable objects, i.e. call
x.__del__, call x2.__del__ (and maybe more)
=> x2 is reachable again => free memory of x; let x2 survive
(x2 even survives at least for another gc-cycle if the finalizer of x or
x2 only created a weak ref)

At least in Java/Jython case I would call x2 to be "resurrected", i.e.
its finalizer was called and weakrefs were cleared. It was on the
death-list and escaped from it. This finally brings the definition of
the word "resurrection" to its limit in language independent sense as
one can argue there was no resurrection of x2 in CPython although it's
one and the same scenario.


In which use case exactly?

The one with "indirect resurrection".
Would it have CPython behavior as sketched above or Java/Jython
behavior? (I confirmed the sketched behavior only for ref-drop-to-zero
triggered cleanup)


Best,

Stefan


On 10/27/2014 04:31 PM, Antoine Pitrou wrote:

On Mon, 27 Oct 2014 16:20:06 +0100
Stefan Richthofer  wrote:

I already pointed out
"One can surely argue x2 has never been dead, or see it as
"it was killed along with x and then resurrected by x"."

In Java and thus in Jython, it is treated as the second one.

You mean Jython deletes instance attributes before calling __del__ ?
That would make most __del__ implementations quite useless...
And actually your own example would fail with an AttributeError on
"X.y = self.z".


What would still be interesting (at least when Jython 3 is born),
is which of the mentioned behaviors occurs if it is
performed by CPython's cyclic gc (consistently the first one I would guess).

In which use case exactly? :-) I've lost track a bit, since you've
posted several examples...

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

2014-10-27 Thread Stefan Richthofer


I already admitted that it is implementation specific whether one would
talk of resurrection, even in one and the same scenario. (Although
I would prefer to agree on an abstract notion of the resurrection term.)


If Jython does things differently, then certainly its behaviour is
incompatible with the common expectations of Python developers.


Guido recently pointed out that it is allowed for different Python
implementations to alter details of gc behavior. (And I suppose this
was more a reminder of already common consensus.)

However I agree that some aspects could be improved and I am looking
at it. So far I have all answers I needed. Thanks for the discussion!


-Stefan





On 10/27/2014 05:36 PM, Antoine Pitrou wrote:

On Mon, 27 Oct 2014 17:23:23 +0100
Stefan Richthofer  wrote:

You mean Jython deletes instance attributes before calling __del__ ?

No. I think the term of "object resurrection" usually does not mean bringing
back a deleted object in the sense that memory was already freed.
I think it rather means that nothing referred to an object, so it was on the
"kill-list" of gc or zero-ref-count macro.

"x2" does *not* have its refcount drop to zero, since it is still
referenced by x. In other words, "x2" can only be on a "kill list"
after "x" has been finalized, which can only be *after* __del__ was
executed.

If Jython does things differently, then certainly its behaviour is
incompatible with the common expectations of Python developers.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

2014-10-27 Thread Stefan Richthofer

>I think 'resuscitation' might be a better metaphor.

The term 'resurrection' is not my invention, but well established:
http://en.wikipedia.org/wiki/Object_resurrection

I well understand why Antoine objects to calling it "resurrection" in CPython 
due to
implementation specific reasons. But in the above article (which I consider 
rather
detailed) I can't find anything stating that an object's ref-count must drop to 
zero
at any time in order to call it "resurrected". In contrast, it clarifies that 
objects
can not only resurrect themselves:
"...which may in turn make that object or another garbage object (reachable 
from the
object with a finalizer) reachable again"
"If this happens, the referenced object – which is not necessarily the finalized
object – is no longer garbage, and cannot be deallocated"

> "x2" does *not* have its refcount drop to zero, since it is still
> referenced by x. In other words, "x2" can only be on a "kill list"
> after "x" has been finalized, which can only be *after* __del__ was
> executed.

x resurrects x2 in the sense that it must "actively" have an action
in its finalizer that establishes a new reference to x2 in non-garbage or
environment memory. Otherwise x as the "final life support link" of x2
would cause x2's ref count *actually* drop to zero in the next step.

I never wanted this to become a discussion about the definition of object
resurrection. I just wanted to understand which details in different
behavior (such as weakref breaking) are okay and which are bugs (as
breaking consistency of id() in Jython).

Regards

-Stefan

> Gesendet: Montag, 27. Oktober 2014 um 23:36 Uhr
> Von: "Terry Reedy" 
> An: python-dev@python.org
> Betreff: Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes 
> break on object resurrection
>
> On 10/27/2014 12:23 PM, Stefan Richthofer wrote:
> >
> >> You mean Jython deletes instance attributes before calling __del__ ?
> >
> > No. I think the term of "object resurrection" usually does not mean
> > bringing
> > back a deleted object in the sense that memory was already freed.
> > I think it rather means that nothing referred to an object, so it was on
> > the
> > "kill-list" of gc or zero-ref-count macro.
> 
> In either case, there is a final reference keeping the object alive, 
> like an hospital patient kept alive by a final link with a life-support 
> machine.  I think 'resuscitation' might be a better metaphor.
> 
> -- 
> Terry Jan Reedy
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] TypeError messages

2015-02-20 Thread Stefan Richthofer

Honestly, the right solution would be to have a function or macro that generates the TypeError messages

from X, Y, Z arguments. (Until now I actually believed this would be already the case)

- type errors would be of uniform style

- if for some reoson the format should be changed, this can be done in one central place

- if someone would want to parse the error message this would be feasible

I suppose it should be straight forward to find error message creations in the source by searching for "TypeError" or something.

Maybe this kind of cleanup could be done along with the implementation of PEP 484?

-Stefan

Gesendet: Freitag, 20. Februar 2015 um 15:05 Uhr
Von: "Brett Cannon"
An: "Serhiy Storchaka" , python-dev@python.org
Betreff: Re: [Python-Dev] TypeError messages

On Thu Feb 19 2015 at 5:52:07 PM Serhiy Storchaka wrote:

Different patterns for TypeError messages are used in the stdlib:

expected X, Y found
expected X, found Y
expected X, but Y found
expected X instance, Y found
X expected, not Y
expect X, not Y
need X, Y found
X is required, not Y
Z must be X, not Y
Z should be X, not Y

and more.

What the pattern is most preferable?

My preference is for "expected X, but found Y".

Some messages use the article before X or Y. Should the article be used
or omitted?

Some messages (only in C) truncate actual type name (%.50s, %.80s,
%.200s, %.500s). Should type name be truncated at all and for how limit?

I assume this is over some worry of string size blowing out memory allocation or something? If someone can show that's an actual worry then fine, otherwise I say don't truncate.

Type names newer truncated in TypeError messages raised in Python code.

Some messages enclose actual type name with single quotes ('%s',
'%.200s'). Should type name be quoted? It is uncommon if type name
contains spaces.

I agree that type names don't need to be quoted.

___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 0-base and 1-base indexed iterables? Custom slicing rules?

2015-03-22 Thread Stefan Richthofer

Pascal allows to define custom indexing. The indexes are defined on array declaration

myArray : Array[1..5] of Integer;

see http://pascal-programming.info/lesson10.php

However, in pascal array-size must be set at compile time, wich is rather annoying, but allows for some optimizations.

 

Visual Basic uses both 0- and 1-based indexing throughout its API. Arrays are always 0-indexed, but there are various

index-taking methods that use 1-indexing. It actually feels like a mess and I never enjoyed using VB, partly (but in

significant part) because of this issue.

 

With some experience in these languages I can tell you, it is really really annoying to figure out the indexing style

each time you work with an array (that you did not create yourself, but was provided by some framework API).

(Even if you created it yourself, it will become hard to keep track when your project grows.)

People *will* abuse this feature if it exists and fragment API into different paradigms. Some migrate from Matlab

and will thus use 1-indexing versus those who "grew up" with numpy or whatever.

Believe it or not, array-indexing is already error-prone enough if you work with nested numpy arrays, apply various

dimension-changing functions, use extensive slicing over several dimensions with customs step-size, transpose

arrays frequently etc. Having to care about indexing style in addition would make it a real nightmare.
 

 

-Stefan


 

Gesendet: Sonntag, 22. März 2015 um 06:46 Uhr
Von: "pedro santos" 
An: python-dev@python.org
Betreff: [Python-Dev] 0-base and 1-base indexed iterables? Custom slicing rules?




Hi, I'm an Art and CG student learning Python and today's exercise was about positions in a tiled room. The fact that I had to check if a position was inside the room and given that in a 1x1 room, 0.0 was considered in and 1.0 was considered out, it kept me thinking about 0-base indexing iterables and slicing.

Read some articles and discussions, some pros and cons to each 0-base and 1-base, concerns about slicing, etc. But ultimately the question that got stuck in me and didn't found an answer was:
Why can't both 0-base and 1-base indexing exist in the same language, and why can't slicing be customized?

 

If I'm indexing the ruler marks, intervals, boundaries, dots, makes sense to start of at 0; rul=[0,1,2,3,4,5,6] would index every mark on my ruler so that accordingly rul[0]=0, rul[5]=5.
If I'm indexing the blue circles, natural number quantities, objects, spans, makes sense to start at 1; cir= [1,2,3,4,5] so that cir[1]=1 and cir[5]=5.

Now, a lot of the discussion was to do with slicing coupled with the indexing and I don't totally understand why.

a ≤ x < b is not so intuitive when dealing with objects ("I want balls 1 up to the the one before 3"), so on one side, you put the finger on what you want and on the other, on what you don't want. But this method does have the neat property of producing neighbor selections that border perfectly, as in [:a][a:b][b:c]. Although in inverse order(-1), the results can be unexpected as it returns values off-by-one from its counterpart like; L=[0,1,2,3,4,5] so that L[1:3]=[1,2] and L[3:1:-1]=[3:2]. So it's consistent with the rule a ≤ x < b, grabbing the lower limit item, but it can feel strange by not producing the same selection with inverse order.

a ≤ x ≤ b is a natural way to select objets ("I want the balls 1 up to 3"), so you're putting the finger on the things you want. If you inverse the order(-1) it's still very easy to grasp what are you picking because whatever you select it's included like: L=[0,1,2,3,4,5] so that L[1:3]=[1,2,3] and L[3:1:-1]=[3,2,1]. Problems seem to arrive though, when trying to do neighbor selections, where one would have to do [:a][a+1:b][b+1:c] to have the border perfectly. That terrible?

Even though one could see a ≤ x < b to be more adept to 0-base, and a ≤ x ≤ b to be more adept to 1-base, the way I see it, index base and slicing rules could be somehow independent. And one would index and slice the way it would fit the rationale of the problem or the data, because even slicing a 1-base indexed array with a ≤ x < b, would still produce an expected outcome, as in cir=[1,2,3,4,5] so that cir[1:3]=[1,2] or cir[:3]=[1,2].
Same thing applying a ≤ x ≤ b to a 0-base indexed array, as in rul[0,1,2,3,4,5] so that rul[:2]=[0,1,2] or rul[0:2]=[0,1,2].

Given that python is an example of human friendly code language, emphasizing readability, wouldn't having 0 and 1 base indexing and custom slicing methods, improve the thought process when writing and reading the code, by fitting them better to specific contexts or data?
Is there some language that provides both or each language picks only one?

Cheers

 

 
--






	
		
			
			
			
			
			Pedro Alpiarça dos Santos 
			Animator  3DModeler  Illustrator 
			>>  http://probiner.x10.mx/
			
		
	


 




___ Python-Dev mailing list Python-Dev@python.org htt

Re: [Python-Dev] Python-versus-CPython question for mul dispatch

2015-05-14 Thread Stefan Richthofer

>>Should I be worried?

You mean should *I* be worried ;)

Stuff like this is highly relevant for JyNI, so thanks very much for clarifying 
this
subtle behavior. It went onto my todo-list right now to ensure that JyNI will 
emulate
this behavior as soon as I am done with gc-support. (Hopefully it will be 
feasible,
but I can only tell in half a year or so since there are currently other 
priorities.)
Still, this "essay" potentially will save me a lot of time.

So, everybody please feel encouraged to post things like this as they come up. 
Maybe
there could be kind of a pitfalls-page somewhere in the docs collecting these 
things.

Best

Stefan


> Gesendet: Freitag, 15. Mai 2015 um 02:45 Uhr
> Von: "Nathaniel Smith" 
> An: "Python Dev" 
> Betreff: [Python-Dev] Python-versus-CPython question for __mul__ dispatch
>
> Hi all,
> 
> While attempting to clean up some of the more squamous aspects of
> numpy's operator dispatch code [1][2], I've encountered a situation
> where the semantics we want and are using are possible according to
> CPython-the-interpreter, but AFAICT ought not to be possible according
> to Python-the-language, i.e., it's not clear to me whether it's
> possible even in principle to implement an object that works the way
> numpy.ndarray does in any other interpreter. Which makes me a bit
> nervous, so I wanted to check if there was any ruling on this.
> 
> Specifically, the quirk we are relying on is this: in CPython, if you do
> 
>   [1, 2] * my_object
> 
> then my_object's __rmul__ gets called *before* list.__mul__,
> *regardless* of the inheritance relationship between list and
> type(my_object). This occurs as a side-effect of the weirdness
> involved in having both tp_as_number->nb_multiply and
> tp_as_sequence->sq_repeat in the C API -- when evaluating "a * b",
> CPython tries a's nb_multiply, then b's nb_multiply, then a's
> sq_repeat, then b's sq_repeat. Since list has an sq_repeat but not an
> nb_multiply, this means that my_object's nb_multiply gets called
> before any list method.
> 
> Here's an example demonstrating how weird this is. list.__mul__ wants
> an integer, and by "integer" it means "any object with an __index__
> method". So here's a class that list is happy to be multiplied by --
> according to the ordinary rules for operator dispatch, in the example
> below Indexable.__mul__ and __rmul__ shouldn't even get a look-in:
> 
> In [3]: class Indexable(object):
>...: def __index__(self):
>...: return 2
>...:
> 
> In [4]: [1, 2] * Indexable()
> Out[4]: [1, 2, 1, 2]
> 
> But, if I add an __rmul__ method, then this actually wins:
> 
> In [6]: class IndexableWithMul(object):
>...: def __index__(self):
>...: return 2
>   ...: def __mul__(self, other):
>...: return "indexable forward mul"
>...: def __rmul__(self, other):
>...: return "indexable reverse mul"
> 
> In [7]: [1, 2] * IndexableWithMul()
> Out[7]: 'indexable reverse mul'
> 
> In [8]: IndexableWithMul() * [1, 2]
> Out[8]: 'indexable forward mul'
> 
> NumPy arrays, of course, correctly define both __index__ method (which
> raises an array on general arrays but coerces to int for arrays that
> contain exactly 1 integer), and also defines an nb_multiply slot which
> accepts lists and performs elementwise multiplication:
> 
> In [9]: [1, 2] * np.array(2)
> Out[9]: array([2, 4])
> 
> And that's all great! Just what we want. But the only reason this is
> possible, AFAICT, is that CPython 'list' is a weird type with
> undocumented behaviour that you can't actually define using pure
> Python code.
> 
> Should I be worried?
> 
> -n
> 
> [1] https://github.com/numpy/numpy/pull/5864
> [2] https://github.com/numpy/numpy/issues/5844
> 
> -- 
> Nathaniel J. Smith -- http://vorpus.org
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
> 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Help/advice needed with JyNI issue #4 (Tkinter on OSX)

2016-04-04 Thread Stefan Richthofer

Hey everybody,

I need help/advice for this JyNI-related issue: 
https://github.com/Stewori/JyNI/issues/4
Especially I need advice from someone familiar with TCL and TK internals, 
preferably also Tkinter.
The issue is rather strange in the sense that it works well on Linux, while the 
program hangs on OSX. Everything we found out so far was collected in the 
thread linked above. Briefly speaking, on OSX TCL/TK does not produce a 
particular event the loop is waiting for and does not display the window. 
However logging suggests that calls to TCL/TK API are identical between Linux 
and OSX runs, so we are really stuck here in finding out what is different on 
Linux (our current logging does not cover function argument values though).
Any advise how I can debug interaction with TCL/TK to find the reason for the 
missing event would be helpful.

(Sorry if you might regard this off-topic for Python-dev; since JyNI is 
somewhat a crossover-project (also containing lots of CPython 2.7 code) I am 
asking in various locations. Starting here, because in this list I see best 
chances to find someone who can help within the Python ecosystem. Next I would 
look for a TCL/TK forum or something.)

Thanks!

Stefan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 4: don't remove anything, don't break backward compatibility

2014-03-10 Thread Stefan Richthofer

I don't see the point in this discussion.
As far as I know, the major version is INTENDED to
indicate backward-incompatible changes.
The meaning of the versioning scheme is literally

[API compatibility].[new features].[bug fixes],

isn't it?

So all you are asking for is never do produce a Python 4.x
This is not necessarily bad, for instance Java has never
increased its major version ever (it is still 1.7.x). An indeed,
it never dropped features, just "deprecated".

> I don't see any reason to bump 
> the major version number until after Python 3.9.

Even then, there is no need for 4.0; you can just have 3.10, 3.11 etc.


Cheers

Stefan



> Gesendet: Montag, 10. März 2014 um 16:04 Uhr
> Von: "Steven D'Aprano" 
> An: python-dev@python.org
> Betreff: Re: [Python-Dev] Python 4: don't remove anything, don't break 
> backward compatibility
>
> On Mon, Mar 10, 2014 at 02:55:26PM +0100, Victor Stinner wrote:
> [...]
> > So can we please try to stop scheduling another major Python version
> > breaking almost all modules and all applications just to be pendantic?
> > 
> > No, we should not remove any old feature in Python 4. Python 4 should
> > be just a minor release following the previous 3.x release.
> 
> I often talk about "Python 4000" as the next possible opportunity for 
> major backwards incompatible changes, but of course that's not decided 
> yet, and given the long term pain of the 2->3 transition, it may be 
> quite conservative, with no radical changes. Perhaps I ought to use 
> Python 5000 as my target for radical language changes?
> 
> 
> > For example, I propose to release the next major Python version (3.5)
> > with the version 4.0 but without removing anything. (It's just an
> > example, it can wait another release.) 
> 
> If Python 4 is a conservative release, I don't see any reason to bump 
> the major version number until after Python 3.9. So, assuming no further 
> radical changes to the language like 2->3, we'd have five more point 
> releases before needing to deal with 4.0.
> 
> Perhaps we need a long-term schedule?
> 
> 3.5: August 2015
> 3.6: February 2017
> 3.7: August 2018
> 3.8: February 2020
> 3.9: August 2021
> 4.0: February 2023
> 
> give or take a few months.
> 
> 
> -- 
> Steven
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 4: don't remove anything, don't break backward compatibility

2014-03-10 Thread Stefan Richthofer

> Guido famously hates two digit minor version numbers. :)

This is no problem either. Simply switch to hexadecimal numbering ;)


> Gesendet: Montag, 10. März 2014 um 16:59 Uhr
> Von: "Barry Warsaw" 
> An: python-dev@python.org
> Betreff: Re: [Python-Dev] Python 4: don't remove anything, don't break 
> backward compatibility
>
> On Mar 10, 2014, at 04:25 PM, Stefan Richthofer wrote:
> 
> >> I don't see any reason to bump 
> >> the major version number until after Python 3.9.
> >
> >Even then, there is no need for 4.0; you can just have 3.10, 3.11 etc.
> 
> Guido famously hates two digit minor version numbers. :)
> 
> -Barry
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-22 Thread Stefan Richthofer

>> I am currently exploring a type hint generator that produces hints out of types used in unit tests.

Note that pytypes (https://github.com/Stewori/pytypes) already supports this. It can dump PEP 484 style stubfiles from runtime type observations (made via decorators or profiler hook).

>> isinstance(obj, T) and issubclass(Cls, T) already fail if T is a subscripted generic like List[int], so that again nothing new here. To check runtime subtyping with such types one can write a third party introspection tool based on typing_inspect package on PyPI

There are public API functions in pytypes for isinstance and issubclass with support for most PEP 484 types, see https://github.com/Stewori/pytypes#is_of_typeobj-cls and https://github.com/Stewori/pytypes#is_subtypesubclass-superclass respectively. We also use them internally for pytypes' runtime typechecking features.

Unfortunately there is no release finalized yet, but it's mostly cleanup work and pip integration left to do. Hope to get a beta release done soon.

Best

-Stefan

Gesendet: Donnerstag, 22. Juni 2017 um 23:53 Uhr
Von: "Ivan Levkivskyi" 
An: "Markus Wissinger" 
Cc: Python-Dev 
Betreff: Re: [Python-Dev] PEP 544: Protocols - second round

On 22 June 2017 at 10:44, Markus Wissinger  wrote:

I have to admit I am not happy with separating the concepts of 'runtime' and 'static' types as implied by pep544.

This is not something new, already PEP 483 makes a clear distinction between types (a static concept) and classes (a runtime concept).

Failing isinstance/issubclass calls for protocols would hurt there. I understand that any type checker code that could provide isinstance functionality for pep544 protocols would rely on method signatures that my hint generator is just producing.

isinstance(obj, T) and issubclass(Cls, T) already fail if T is a subscripted generic like List[int], so that again nothing new here. To check runtime subtyping with such types one can write a third party introspection tool based on typing_inspect package on PyPI (which potentially might in future become an official wrapper for currently internal typing API).

--

Ivan

___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] sum(...) limitation

[Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection

Re: [Python-Dev] TypeError messages

Re: [Python-Dev] 0-base and 1-base indexed iterables? Custom slicing rules?

Re: [Python-Dev] Python-versus-CPython question for mul dispatch

[Python-Dev] Help/advice needed with JyNI issue #4 (Tkinter on OSX)

Re: [Python-Dev] Python 4: don't remove anything, don't break backward compatibility

Re: [Python-Dev] Python 4: don't remove anything, don't break backward compatibility

Re: [Python-Dev] PEP 544: Protocols - second round

16 matches

Site Navigation

Mail list logo

Footer information