Re: [Python-Dev] gc ideas -- sparse memory

2010-12-05 Thread Greg Ewing
Martin v. Löwis wrote: The hole C API would break if objects would move in memory. Since they have to stay at fixed addresses, it's easy enough to use the address as ID. Yes. Some of the discussion here seems to be assuming that the reason Python doesn't move objects is so that it can use the a

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-05 Thread Stephen J. Turnbull
"Martin v. Löwis" writes: > > Why is useful to expose an identity hash? AFAICS it is *only* useful > > in building an identity hash table. If so, why not just provide id() > > or the is operator or both and be done with it? > > That's precisely James' point: Java provides the identity hash

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Martin v. Löwis
> Why is useful to expose an identity hash? AFAICS it is *only* useful > in building an identity hash table. If so, why not just provide id() > or the is operator or both and be done with it? That's precisely James' point: Java provides the identity hash *instead* of the id() function (i.e. it d

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Stephen J. Turnbull
Quotes are out of order. "Martin v. Löwis" writes: > No, it's not. java.lang.Object.hashCode() is equivalent to Python's > hash(). In particular, for both of these, the following requirement > holds: object that *compare* equal must hash equal. Aha. I see, now. That is indeed a different co

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Maciej Fijalkowski
On Sat, Dec 4, 2010 at 1:37 PM, Antoine Pitrou wrote: > On Sat, 4 Dec 2010 15:06:45 +1100 > Cameron Simpson wrote: >> On 03Dec2010 18:15, James Y Knight wrote: >> | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: >> | > gc is implementation specific. CPython uses ref counting + cycle >> | > gc. A

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Stephen J. Turnbull
Steven D'Aprano writes: > Martin v. Löwis wrote: > >> It seems counter-productive to me to bother with an identity function > >> which doesn't meet that constraint. If id(x) == id(y) implies nothing > >> about x and y (they may, or may not, be the same object) then what's the > >> point? > >

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Martin v. Löwis
> Surely even in Java or C#, objects have an *identity* even if the > language doesn't provide a way to query their distinct *identification*. When people said "the id of an object should this or that", they always meant identification, not identity (perhaps except for you). Nobody would propose t

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Antoine Pitrou
On Sat, 4 Dec 2010 15:06:45 +1100 Cameron Simpson wrote: > On 03Dec2010 18:15, James Y Knight wrote: > | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: > | > gc is implementation specific. CPython uses ref counting + cycle > | > gc. A constraint on all implementations is that objects have a fixed

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Steven D'Aprano
Martin v. Löwis wrote: I'm afraid I don't follow you. Unless you're suggesting some sort of esoteric object system whereby objects *don't* have identity (e.g. where objects are emergent properties of some sort of distributed, non-localised "information"), any object naturally has an identity -- i

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Martin v. Löwis
> I'm afraid I don't follow you. Unless you're suggesting some sort of > esoteric object system whereby objects *don't* have identity (e.g. where > objects are emergent properties of some sort of distributed, > non-localised "information"), any object naturally has an identity -- > itself. Not in

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-04 Thread Martin v. Löwis
> Am I still missing something? Apparently. The hole C API would break if objects would move in memory. Since they have to stay at fixed addresses, it's easy enough to use the address as ID. There is no problem to be solved here. Regards, Martin ___ Pyt

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Steven D'Aprano
James Y Knight wrote: On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote: On 12/3/2010 7:46 PM, James Y Knight wrote: Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this using "identityHashCode()"), you can still make

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Maciej Fijalkowski
On Sat, Dec 4, 2010 at 6:34 AM, James Y Knight wrote: > On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote: >> On 12/3/2010 7:46 PM, James Y Knight wrote: >> >>> Sure they are. This is what Java provides you, for example. If you >>> have fixed, but potentially non-unique ids (in Java you get this >>>

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Dima Tisnek
On 3 December 2010 16:45, "Martin v. Löwis" wrote: >> Oh my bad, I must've confused python with some research paper. >> Unique id is not so hard to make without an address. >> >> While on this topic, what is the real need for unique ids? > > They are absolutely needed for mutable objects. For immu

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Terry Reedy
On 12/3/2010 11:06 PM, Cameron Simpson wrote: On 03Dec2010 18:15, James Y Knight wrote: | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: |> gc is implementation specific. CPython uses ref counting + cycle |> gc. A constraint on all implementations is that objects have a fixed, |> unique id du

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread James Y Knight
On Dec 3, 2010, at 10:50 PM, Terry Reedy wrote: > On 12/3/2010 7:46 PM, James Y Knight wrote: > >> Sure they are. This is what Java provides you, for example. If you >> have fixed, but potentially non-unique ids (in Java you get this >> using "identityHashCode()"), you can still make an identity >

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Cameron Simpson
On 03Dec2010 18:15, James Y Knight wrote: | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: | > gc is implementation specific. CPython uses ref counting + cycle | > gc. A constraint on all implementations is that objects have a fixed, | > unique id during their lifetime. CPython uses the address as

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Terry Reedy
On 12/3/2010 7:46 PM, James Y Knight wrote: Sure they are. This is what Java provides you, for example. If you have fixed, but potentially non-unique ids (in Java you get this using "identityHashCode()"), you can still make an identity I do not see the point of calling a (non-unique) hash valu

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread James Y Knight
On Dec 3, 2010, at 7:05 PM, Terry Reedy wrote: > I left out that the id must be an int. > >> It's somewhat unfortuante that python has this constraint, instead of >> the looser: "objects have a fixed id during their lifetime", which is >> much easier to implement, and practically as useful. > > G

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Terry Reedy
On 12/3/2010 6:15 PM, James Y Knight wrote: On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: gc is implementation specific. CPython uses ref counting + cycle gc. A constraint on all implementations is that objects have a fixed, unique id during their lifetime. CPython uses the address as the id, s

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Martin v. Löwis
> Oh my bad, I must've confused python with some research paper. > Unique id is not so hard to make without an address. > > While on this topic, what is the real need for unique ids? They are absolutely needed for mutable objects. For immutable ones, it would be ok to claim that they are identica

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Dima Tisnek
Oh my bad, I must've confused python with some research paper. Unique id is not so hard to make without an address. While on this topic, what is the real need for unique ids? Also I reckon not all objects need a unique id like this, e.g. interned strings, simple data types and hashable and compara

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Martin v. Löwis
Am 03.12.2010 23:55, schrieb Dima Tisnek: > How hard or reasonable would it be to free memory pages on OS level? Very easy. Python already does that. > [pcmiiw] Gabage collection within a generation involves moving live > objects to compact the generation storage. This separates the memory > regi

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread James Y Knight
On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: > gc is implementation specific. CPython uses ref counting + cycle gc. A > constraint on all implementations is that objects have a fixed, unique id > during their lifetime. CPython uses the address as the id, so it cannot move > objects. Other impl

Re: [Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Terry Reedy
On 12/3/2010 5:55 PM, Dima Tisnek wrote: How hard or reasonable would it be to free memory pages on OS level? [pcmiiw] Gabage collection within a generation involves moving live objects to compact the generation storage. This separates the memory region into 2 parts "live" and "cleared", the poi

[Python-Dev] gc ideas -- sparse memory

2010-12-03 Thread Dima Tisnek
How hard or reasonable would it be to free memory pages on OS level? [pcmiiw] Gabage collection within a generation involves moving live objects to compact the generation storage. This separates the memory region into 2 parts "live" and "cleared", the pointer to the beginning of the "cleared" part