[Python-Dev] py2.7: dictobject not properly resizing

2013-03-30 Thread Micha Gorelick
I was taking a look at dictobject.c and realized that the logic controlling whether a resizedict will occur in dict_set_item_by_hash_or_entry disallows for the shrinking of a dictionary. This is contrary to what the comments directly above say: (http://hg.python.org/cpython/file/f3032825f637/Obje

Re: [Python-Dev] py2.7: dictobject not properly resizing

2013-03-30 Thread Micha Gorelick
True, but your example mechanism of getting a shrink event is purely based on ma_fill. This is happening because your last loop is increasing ma_fill to the point where it thinks it needs to resize because of the load factor and then it calculates the new size based on ma_used. The comment that I

Re: [Python-Dev] py2.7: dictobject not properly resizing

2013-03-31 Thread Micha Gorelick
So I did a bit of benchmarking and attached is the code I used. With downsizing happening when ma_used * 2 <= ma_filled, or the following for the condition in question: if (mp->ma_used <= n_used || (mp->ma_fill*3 < (mp->ma_mask+1)*2 && mp->ma_used*2 > mp->ma_fill)) I see marginally faster pe