Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-26 Thread Kristján Valur Jónsson
> -Original Message- > From: "Martin v. Löwis" [mailto:mar...@v.loewis.de] > > This is easy in a debug build, using sys.getobjects(). In a release build, > you can > use pympler: > > start = pympler.muppy.get_size(pympler.muppy.get_objects()) > run_complicated_tests() > end = pympler.m

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-26 Thread Kristján Valur Jónsson
t; From: Nick Coghlan [mailto:ncogh...@gmail.com] > Sent: 24. apríl 2012 11:42 > To: Kristján Valur Jónsson > Cc: R. David Murray; Antoine Pitrou; python-dev@python.org > Subject: Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing > dictionaries (closes #13903) > > On Tue, Apr 2

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-25 Thread Martin v. Löwis
Benchmarks should measure memory usage too, of course. Sadly that is not possible in standard cPython. It's actually very easy in standard CPython, using sys.getsizeof. Yes, you can query each python object about how big it thinks it is. What I'm speaking of is more like: start_allocs, start_

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-25 Thread Kristján Valur Jónsson
> -Original Message- > Take a look at the benchmark suite at > http://hg.python.org/benchmarks/ > The test runner has an -m option that profiles memory usage, you could take > a look at how that is implemented > Yes, out of process monitoring of memory as reported by the OS. We do gathe

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-25 Thread Mark Shannon
: Implement PEP 412: Key-sharing dictionaries (closes #13903) Benchmarks should measure memory usage too, of course. Sadly that is not possible in standard cPython. It's actually very easy in standard CPython, using sys.getsizeof. Yes, you can query each python object about how big it t

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-25 Thread Kristján Valur Jónsson
python: Implement PEP 412: Key-sharing > dictionaries (closes #13903) > > > Benchmarks should measure memory usage too, of course. Sadly that is > > not possible in standard cPython. > > It's actually very easy in standard CPython, using sys.getsizeof. > Yes, you ca

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-24 Thread martin
Benchmarks should measure memory usage too, of course. Sadly that is not possible in standard cPython. It's actually very easy in standard CPython, using sys.getsizeof. Btw, this is of great interest to me at the moment, our Shanghai engineers are screaming at the memory waste incurred by d

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-24 Thread Nick Coghlan
On Tue, Apr 24, 2012 at 8:24 PM, Kristján Valur Jónsson wrote: > Perhaps I should write about this on my blog.  Updating the memory allocation > macro layer in > cPython for embedding is something I'd be inclined to contribute, but it will > involve a large amount > of bikeshedding, I'm sure :)

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-24 Thread Antoine Pitrou
On Tue, 24 Apr 2012 10:24:16 + Kristján Valur Jónsson wrote: > > Btw, this is of great interest to me at the moment, our Shanghai engineers > are screaming at the > memory waste incurred by dictionaries. A 10 item dictionary consumes 1/2k on > 32 bits, did you > know this? The sparseness

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-24 Thread Kristján Valur Jónsson
n.org > Subject: Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing > dictionaries (closes #13903) > > On Mon, 23 Apr 2012 22:22:18 +0200, Antoine Pitrou > wrote: > > On Mon, 23 Apr 2012 17:24:57 +0200 > > benjamin.peterson wrote: > > > http://hg.python.

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-23 Thread R. David Murray
On Mon, 23 Apr 2012 22:22:18 +0200, Antoine Pitrou wrote: > On Mon, 23 Apr 2012 17:24:57 +0200 > benjamin.peterson wrote: > > http://hg.python.org/cpython/rev/6e5855854a2e > > changeset: 76485:6e5855854a2e > > user:Benjamin Peterson > > date:Mon Apr 23 11:24:50 2012 -0400 > > s

Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

2012-04-23 Thread Antoine Pitrou
On Mon, 23 Apr 2012 17:24:57 +0200 benjamin.peterson wrote: > http://hg.python.org/cpython/rev/6e5855854a2e > changeset: 76485:6e5855854a2e > user:Benjamin Peterson > date:Mon Apr 23 11:24:50 2012 -0400 > summary: > Implement PEP 412: Key-sharing dictionaries (closes #13903)