Re: [Python-Dev] A new dictionary implementation

2012-02-17 Thread Mark Shannon
On 15/02/12 21:09, Yury Selivanov wrote: Hello Mark, First, I've back-ported your patch on python 3.2.2 (which was relatively easy). Almost all tests pass, and those that don't are always failing on my machine if I remember. The patch can be found here: http://goo.gl/nSzzY Then, I compared me

Re: [Python-Dev] A new dictionary implementation

2012-02-15 Thread Yury Selivanov
Hello Mark, First, I've back-ported your patch on python 3.2.2 (which was relatively easy). Almost all tests pass, and those that don't are always failing on my machine if I remember. The patch can be found here: http://goo.gl/nSzzY Then, I compared memory footprint of one of our applications (

Re: [Python-Dev] A new dictionary implementation

2012-02-15 Thread Antoine Pitrou
On Mon, 13 Feb 2012 12:31:38 + Mark Shannon wrote: > Note that the json benchmark is unstable and should be ignored. Can you elaborate? If it's unstable it should be fixed, not ignored :) Also, there are two different mako results in your message, which one is the right one? Thanks Antoine

Re: [Python-Dev] A new dictionary implementation

2012-02-15 Thread Mark Shannon
Any opinions on my new dictionary implementation? I'm happy to take silence on the PEP as tacit approval, but the code definitely needs reviewing. Issue: http://bugs.python.org/issue13903 PEP: https://bitbucket.org/markshannon/cpython_new_dict/src/6c4d5d9dfc6d/pep-new-dict.txt Repository https

Re: [Python-Dev] A new dictionary implementation

2012-02-09 Thread Mark Shannon
francis wrote: Hi Mark, Bah... typo in assert statement. My fault for not testing the debug build (release build worked fine). Both builds working now. Yeah, now is working and passes all tests also on my machine. I've tried to run the test suite but I'm getting a SyntaxError: (may be you know

Re: [Python-Dev] A new dictionary implementation

2012-02-09 Thread francis
Hi Mark, Bah... typo in assert statement. My fault for not testing the debug build (release build worked fine). Both builds working now. Yeah, now is working and passes all tests also on my machine. I've tried to run the test suite but I'm getting a SyntaxError: (may be you know it's just the f

Re: [Python-Dev] A new dictionary implementation

2012-02-09 Thread Michael Foord
On 08/02/2012 15:16, Mark Shannon wrote: Hi, Version 2 is now available. Version 2 makes as few changes to tunable constants as possible, and generally does not change iteration order (so repr() is unchanged). All tests pass (the only changes to tests are for sys.getsizeof() ). Repository: h

Re: [Python-Dev] A new dictionary implementation

2012-02-09 Thread Mark Shannon
francis wrote: Hi Mark, I've just cloned : Repository: https://bitbucket.org/markshannon/cpython_new_dict Do please try it on your machine(s). that's a: Linux random 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64 GNU/Linux and I'm getting: gcc -pthread -c -Wno-unused-resu

Re: [Python-Dev] A new dictionary implementation

2012-02-08 Thread francis
Hi Mark, I've just cloned : Repository: https://bitbucket.org/markshannon/cpython_new_dict Do please try it on your machine(s). that's a: Linux random 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64 GNU/Linux and I'm getting: gcc -pthread -c -Wno-unused-result -g -O0 -Wall

Re: [Python-Dev] A new dictionary implementation

2012-02-08 Thread francis
Just more info: changeset is: 74843:20702d1acf17 Cheers, francis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archiv

Re: [Python-Dev] A new dictionary implementation

2012-02-08 Thread Mark Shannon
Hi, Version 2 is now available. Version 2 makes as few changes to tunable constants as possible, and generally does not change iteration order (so repr() is unchanged). All tests pass (the only changes to tests are for sys.getsizeof() ). Repository: https://bitbucket.org/markshannon/cpython_n

Re: [Python-Dev] A new dictionary implementation

2012-02-02 Thread Mark Shannon
Just a quick update. I've been analysing and profile the behaviour of my new dict and messing about with various implementation options. I've settled on a new implementation. Its the same basic idea, but with better locality of reference for unshared keys. Guido asked: > Another question:

Re: [Python-Dev] A new dictionary implementation

2012-02-02 Thread Martin v. Löwis
Am 02.02.2012 12:30, schrieb Chris Withers: > On 01/02/2012 17:50, Guido van Rossum wrote: >> Another question: a common pattern is to use (immutable) class >> variables as default values for instance variables, and only set the >> instance variables once they need to be different. Does such a clas

Re: [Python-Dev] A new dictionary implementation

2012-02-02 Thread Antoine Pitrou
On Wed, 1 Feb 2012 09:50:55 -0800 Guido van Rossum wrote: > On Wed, Feb 1, 2012 at 9:13 AM, Hans Mulder wrote: > > On 30/01/12 00:30:14, Steven D'Aprano wrote: > >> > >> Mark Shannon wrote: > >>> > >>> Antoine Pitrou wrote: > > > > [..] > > > >>> Antoine is right. It is a reorganisation of th

Re: [Python-Dev] A new dictionary implementation

2012-02-02 Thread Michael Foord
On 02/02/2012 11:30, Chris Withers wrote: On 01/02/2012 17:50, Guido van Rossum wrote: Another question: a common pattern is to use (immutable) class variables as default values for instance variables, and only set the instance variables once they need to be different. Does such a class benefit

Re: [Python-Dev] A new dictionary implementation

2012-02-02 Thread Chris Withers
On 01/02/2012 17:50, Guido van Rossum wrote: Another question: a common pattern is to use (immutable) class variables as default values for instance variables, and only set the instance variables once they need to be different. Does such a class benefit from your improvement? A less common patt

Re: [Python-Dev] A new dictionary implementation

2012-02-01 Thread martin
Hey, I like this! It's a subtle encouragement for developers to initialize all their instance variables in their __init__ or __new__ method, with a (modest) performance improvement for a carrot. (Though I have to admit I have no idea how you do it. Wouldn't the set of dict keys be different while

Re: [Python-Dev] A new dictionary implementation

2012-02-01 Thread Alex
Guido van Rossum python.org> writes: > Hey, I like this! It's a subtle encouragement for developers to > initialize all their instance variables in their __init__ or __new__ > method, with a (modest) performance improvement for a carrot. (Though > I have to admit I have no idea how you do it. Wou

Re: [Python-Dev] A new dictionary implementation

2012-02-01 Thread Guido van Rossum
On Wed, Feb 1, 2012 at 9:13 AM, Hans Mulder wrote: > On 30/01/12 00:30:14, Steven D'Aprano wrote: >> >> Mark Shannon wrote: >>> >>> Antoine Pitrou wrote: > > [..] > >>> Antoine is right. It is a reorganisation of the dict, plus a couple of >>> changes to typeobject.c and object.c to ensure tha

Re: [Python-Dev] A new dictionary implementation

2012-02-01 Thread Hans Mulder
On 30/01/12 00:30:14, Steven D'Aprano wrote: Mark Shannon wrote: Antoine Pitrou wrote: [..] Antoine is right. It is a reorganisation of the dict, plus a couple of changes to typeobject.c and object.c to ensure that instance dictionaries do indeed share keys arrays. I don't quite follow

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread francis
I still have gdb 6.somthing, would you mail me the full output please, so I can see what the problem is. It's done, let me know if you need more output. Cheers, francis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Steven D'Aprano
Mark Shannon wrote: Antoine Pitrou wrote: On Sun, 29 Jan 2012 09:56:11 -0500 Benjamin Peterson wrote: 2012/1/29 Mark Shannon : Hi, Now that issue 13703 has been largely settled, I want to propose my new dictionary implementation again. It is a little more polished than before. If you're se

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Mark Shannon
Matt Joiner wrote: Mark, Good luck with getting this in, I'm also hopeful about coroutines, maybe after pushing your dict optimization your coroutine implementation will get more consideration. Shush, don't say the C word or you'll put people off ;) I'm actually not that fussed about the coro

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Martin v. Löwis
>> Please clarify the status of that code: are you actually proposing >> 6a21f3b35e20 for inclusion into Python as-is? If so, please post it >> as a patch to the tracker, as it will need to be reviewed (possibly >> with requests for further changes). > > I thought it already was a patch. What do I

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Mark Shannon
Martin v. Löwis wrote: Now that issue 13703 has been largely settled, I want to propose my new dictionary implementation again. It is a little more polished than before. Please clarify the status of that code: are you actually proposing 6a21f3b35e20 for inclusion into Python as-is? If so, pleas

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Mark Shannon
francis wrote: On 01/29/2012 11:31 AM, Mark Shannon wrote: It passes all the tests. (I had to change a couple that relied on dict repr() ordering) Hi Mark, I've cloned the repo, build it the I've tried with ./python -m test. I got some errors: First in general: 340 tests OK. 2 tests failed:

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Martin v. Löwis
> Now that issue 13703 has been largely settled, > I want to propose my new dictionary implementation again. > It is a little more polished than before. Please clarify the status of that code: are you actually proposing 6a21f3b35e20 for inclusion into Python as-is? If so, please post it as a patch

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread francis
On 01/29/2012 11:31 AM, Mark Shannon wrote: It passes all the tests. (I had to change a couple that relied on dict repr() ordering) Hi Mark, I've cloned the repo, build it the I've tried with ./python -m test. I got some errors: First in general: 340 tests OK. 2 tests failed: test_dis te

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Mark Shannon
Antoine Pitrou wrote: On Sun, 29 Jan 2012 09:56:11 -0500 Benjamin Peterson wrote: 2012/1/29 Mark Shannon : Hi, Now that issue 13703 has been largely settled, I want to propose my new dictionary implementation again. It is a little more polished than before. If you're serious about changing

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Mark Shannon
Antoine Pitrou wrote: Hi, On Sun, 29 Jan 2012 10:31:48 + Mark Shannon wrote: Now that issue 13703 has been largely settled, I want to propose my new dictionary implementation again. It is a little more polished than before. https://bitbucket.org/markshannon/hotpy_new_dict I briefly took

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Benjamin Peterson
2012/1/29 Antoine Pitrou : > On Sun, 29 Jan 2012 09:56:11 -0500 > Benjamin Peterson wrote: > >> 2012/1/29 Mark Shannon : >> > Hi, >> > >> > Now that issue 13703 has been largely settled, >> > I want to propose my new dictionary implementation again. >> > It is a little more polished than before. >

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Antoine Pitrou
On Sun, 29 Jan 2012 09:56:11 -0500 Benjamin Peterson wrote: > 2012/1/29 Mark Shannon : > > Hi, > > > > Now that issue 13703 has been largely settled, > > I want to propose my new dictionary implementation again. > > It is a little more polished than before. > > If you're serious about changing t

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Benjamin Peterson
2012/1/29 Mark Shannon : > Hi, > > Now that issue 13703 has been largely settled, > I want to propose my new dictionary implementation again. > It is a little more polished than before. If you're serious about changing the dictionary implementation, I think you should write a PEP. It should explai

Re: [Python-Dev] A new dictionary implementation

2012-01-29 Thread Antoine Pitrou
Hi, On Sun, 29 Jan 2012 10:31:48 + Mark Shannon wrote: > > Now that issue 13703 has been largely settled, > I want to propose my new dictionary implementation again. > It is a little more polished than before. > > https://bitbucket.org/markshannon/hotpy_new_dict I briefly took a look at y

[Python-Dev] A new dictionary implementation

2012-01-29 Thread Mark Shannon
Hi, Now that issue 13703 has been largely settled, I want to propose my new dictionary implementation again. It is a little more polished than before. https://bitbucket.org/markshannon/hotpy_new_dict Object-oriented benchmarks use considerably less memory and are sometimes faster (by a small am