Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Andrew MacIntyre
M.-A. Lemburg wrote: > On 2008-02-07 14:09, Andrew MacIntyre wrote: >> Probably in response to the same stimulus as Christian it occurred to me >> that the freelist approach had been adopted long before PyMalloc was >> enabled as standard (in 2.3), and that much of the performance gains >> between

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread M.-A. Lemburg
On 2008-02-08 08:21, Martin v. Löwis wrote: >> One of the hopes of having a custom allocator for Python was to be >> able to get rid off all free lists. For some reason that never happened. >> Not sure why. People were probably too busy with adding new >> features to the language at the time ;-) >

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Christian Heimes
Andrew MacIntyre wrote: > However, my tests do show that something is funny with the current > freelist implementation for floats on at least 2 platforms, and that > doing without that sort of optimisation for float objects would likely > not be a hardship with PyMalloc. float objects are slightly

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Christian Heimes
Andrew MacIntyre wrote: > For the int case, that patch is slower than no free list at all. For the > float case its very slightly faster (55.3s vs 55.5s) than no free list at > all. Slower than the trunk code for both cases. Did you run the test > scripts on your own machine? I've used a simple

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Neal Norwitz
On Feb 8, 2008 10:54 AM, Christian Heimes <[EMAIL PROTECTED]> wrote: > Andrew MacIntyre wrote: > > However, my tests do show that something is funny with the current > > freelist implementation for floats on at least 2 platforms, and that > > doing without that sort of optimisation for float object

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Christian Heimes
Tim Peters wrote: > pymalloc ensures 8-byte alignment. This is one plausible reason to > keep the current int free list: an int object struct holds 3 4-byte > members on most boxes (type pointer, refcount, and the int's value), > and the int freelist code uses exactly 12 bytes for each on most >

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Christian Heimes
Neal Norwitz wrote: > It's not just size. Architectures may require data aligned on 4, 8, > or 16 addresses for optimal performance depending on data type. IIRC, > malloc aligns by 8 (not sure if that was a particular arch or very > common). I don't know if pymalloc handles alignment. Yes, pyma

Re: [Python-Dev] Initial attempt to PyCon sprint tutorial slides are up

2008-02-08 Thread Brett Cannon
On Feb 6, 2008 4:46 AM, Facundo Batista <[EMAIL PROTECTED]> wrote: > 2008/2/4, Brett Cannon <[EMAIL PROTECTED]>: > > > The 1 MB PDF can be found at > > http://www.cs.ubc.ca/~drifty/pycon/sprint_tutorial.pdf . If you find > > any bad info or some info that is really lacking, let me know. But > > Bre

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread M.-A. Lemburg
On 2008-02-08 12:23, M.-A. Lemburg wrote: > On 2008-02-08 08:21, Martin v. Löwis wrote: >>> One of the hopes of having a custom allocator for Python was to be >>> able to get rid off all free lists. For some reason that never happened. >>> Not sure why. People were probably too busy with adding new

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread M.-A. Lemburg
On 2008-02-08 19:28, Christian Heimes wrote: >> In addition to the pure performance aspect, there is the issue of memory >> utilisation. The current trunk code running the int test case in my >> original post peaks at 151MB according to top on my FreeBSD box, dropping >> back to about 62MB after t

Re: [Python-Dev] Initial attempt to PyCon sprint tutorial slides are up

2008-02-08 Thread Brett Cannon
On Feb 8, 2008 12:53 PM, Christian Heimes <[EMAIL PROTECTED]> wrote: > Brett Cannon wrote: > > The corrected version of the slides are now up at the same location. > > I found a minor mistake > > PC/ > * Build files for compilers older than VS 7.1. > > The PC directory contains build directories f

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Andrew MacIntyre
M.-A. Lemburg wrote: > As you can see, Integers and floats fall into the same pymalloc size > class. What's strange in Andrew's result is that both integers > and floats use the same free list technique and fall into the same > pymalloc size class, yet the results are different. My take is not th

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Martin v. Löwis
> Given the background information Python's long implementation could > probably optimized. In 3.0 a long has 3 4-byte members (ref count, > ob_size and *ob_type) plus one to many unsigned shorts (2 bytes each) to > hold the value. If pymalloc aligns the objects at 8 byte address > boundaries would

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Andrew MacIntyre
Tim Peters wrote: > pymalloc ensures 8-byte alignment. This is one plausible reason to > keep the current int free list: an int object struct holds 3 4-byte > members on most boxes (type pointer, refcount, and the int's value), > and the int freelist code uses exactly 12 bytes for each on most >

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Christian Heimes
Martin v. Löwis wrote: > You mean, as the digits? You would have to rewrite the entire long > datatype, which is a tedious exercise. Plus, I believe you would have > to make some operations 64-bit on all systems: when you multiply > digits today, the product will be 32-bit. If you extend the digits

[Python-Dev] Summary of Tracker Issues

2008-02-08 Thread Tracker
ACTIVITY SUMMARY (02/01/08 - 02/08/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1691 open (+30) / 12209 closed (+18) / 13900 total (+48) Open issues with patches: 437 Average durati

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Tim Peters
[Neal Norwitz] > It's not just size. Architectures may require data aligned on 4, 8, > or 16 addresses for optimal performance depending on data type. IIRC, > malloc aligns by 8 (not sure if that was a particular arch or very > common). Just very common. Because malloc has no idea what the poin

Re: [Python-Dev] int/float freelists vs pymalloc

2008-02-08 Thread Andrew MacIntyre
Christian Heimes wrote: > Andrew MacIntyre wrote: >> For the int case, that patch is slower than no free list at all. For the >> float case its very slightly faster (55.3s vs 55.5s) than no free list at >> all. Slower than the trunk code for both cases. Did you run the test >> scripts on your ow

Re: [Python-Dev] Initial attempt to PyCon sprint tutorial slides are up

2008-02-08 Thread Christian Heimes
Brett Cannon wrote: > The corrected version of the slides are now up at the same location. I found a minor mistake PC/ * Build files for compilers older than VS 7.1. The PC directory contains build directories for VC6.0, VC 7.1, VC8.0 and OS2. Christian