Re: [Python-Dev] Compact ordered set

2019-03-07 Thread Inada Naoki
ordered set 2 Hi, Raymond. Thank you for your detailed response and I'm sorry about the late reply. All of your points make sense to me. My implementation has not been battle-tested yet. As I wrote in a previous mail, there is only one benchmark in pyperformance was affected significantly. (M

Re: [Python-Dev] Compact ordered set

2019-02-28 Thread Greg Ewing
Antoine Pitrou wrote: On a more abstract level, set and dict are both content-addressed collections parametered on hash and equality functions. Indeed. It's been said that a set is like "half a dict", and this is why sets were implemented using dicts in the old days. It's kind of an obvious th

Re: [Python-Dev] Compact ordered set

2019-02-28 Thread Chris Angelico
On Thu, Feb 28, 2019 at 10:58 PM Antoine Pitrou wrote: > Some of them may be coming from C++, where the respective > characteristics of set and map (or unordered_set and > unordered_multimap) are closely related. I'm sure other languages > show similar analogies. > > On a more abstract level, set

Re: [Python-Dev] Compact ordered set

2019-02-28 Thread Xavier Morel
> On 2019-02-28, at 12:56 , Antoine Pitrou wrote: > > On Thu, 28 Feb 2019 22:43:04 +1100 > Steven D'Aprano wrote: >> On Wed, Feb 27, 2019 at 02:15:53PM -0800, Barry Warsaw wrote: >> >>> I’m just relaying a data point. Some Python folks I’ve worked with do >>> make the connection between dict

Re: [Python-Dev] Compact ordered set

2019-02-28 Thread Antoine Pitrou
On Thu, 28 Feb 2019 22:43:04 +1100 Steven D'Aprano wrote: > On Wed, Feb 27, 2019 at 02:15:53PM -0800, Barry Warsaw wrote: > > > I’m just relaying a data point. Some Python folks I’ve worked with do > > make the connection between dicts and sets, and have questions about > > the ordering guaran

Re: [Python-Dev] Compact ordered set

2019-02-28 Thread Steven D'Aprano
On Wed, Feb 27, 2019 at 02:15:53PM -0800, Barry Warsaw wrote: > I’m just relaying a data point. Some Python folks I’ve worked with do > make the connection between dicts and sets, and have questions about > the ordering guarantees of then (and how they relate). Sets and dicts are not related b

Re: [Python-Dev] Compact ordered set

2019-02-28 Thread INADA Naoki
On Thu, Feb 28, 2019 at 7:23 AM Henry Chen wrote: > If sets were ordered, then what ought pop() return - first, last, or > nevertheless an arbitrary element? I lean toward arbitrary because in > existing code, set.pop often implies that which particular element is > immaterial. > > dict.popitem()

Re: [Python-Dev] Compact ordered set

2019-02-27 Thread Henry Chen
If sets were ordered, then what ought pop() return - first, last, or nevertheless an arbitrary element? I lean toward arbitrary because in existing code, set.pop often implies that which particular element is immaterial. On Wed, Feb 27, 2019 at 2:18 PM Barry Warsaw wrote: > On Feb 27, 2019, at 1

Re: [Python-Dev] Compact ordered set

2019-02-27 Thread Barry Warsaw
On Feb 27, 2019, at 13:54, Chris Barker via Python-Dev wrote: > > A mapping and a set type really don't have much to do with each other other > than implementation -- anyone that isn't familiar with python C code, or hash > tables in general, wouldn't likely have any expectation of them having

Re: [Python-Dev] Compact ordered set

2019-02-27 Thread Chris Barker via Python-Dev
On Tue, Feb 26, 2019 at 3:43 PM Barry Warsaw wrote: > The behavior differences between dicts and sets is already surprising to > many users, so we should be careful not to make the situation worse. > It's a nice to have, but other than the fact that we all used to use a dict when we really want

Re: [Python-Dev] Compact ordered set

2019-02-26 Thread Barry Warsaw
On Feb 26, 2019, at 13:02, Raymond Hettinger wrote: > * I gave up on ordering right away. If we care about performance, keys can > be stored in the order added; but no effort should be expended to maintain > order if subsequent deletions occur. Likewise, to keep set-to-set operations > effi

Re: [Python-Dev] Compact ordered set

2019-02-26 Thread Victor Stinner
Le mar. 26 févr. 2019 à 17:33, INADA Naoki a écrit : > My company gives me dedicated Linux machine with Core(TM) i7-6700. > So I think it's not issue of my machine. Oh great :-) > perf shows this line caused many page fault. > https://github.com/python/cpython/blob/c606a9cbd48f69d3f4a09204c781dd

Re: [Python-Dev] Compact ordered set

2019-02-26 Thread Raymond Hettinger
Quick summary of what I found when I last ran experiments with this idea: * To get the same lookup performance, the density of index table would need to go down to around 25%. Otherwise, there's no way to make up for the extra indirection and the loss of cache locality. * There was a small win

Re: [Python-Dev] Compact ordered set

2019-02-26 Thread Raymond Hettinger
> On Feb 26, 2019, at 3:30 AM, INADA Naoki wrote: > > I'm working on compact and ordered set implementation. > It has internal data structure similar to new dict from Python 3.6. > > On Feb 26, 2019, at 3:30 AM, INADA Naoki wrote: > > I'm working on compact and ordered set implementation.

Re: [Python-Dev] Compact ordered set

2019-02-26 Thread INADA Naoki
On Wed, Feb 27, 2019 at 12:37 AM Victor Stinner wrote: > > Le mar. 26 févr. 2019 à 12:33, INADA Naoki a écrit : > > - unpickle_list: 8.48 us +- 0.09 us -> 12.8 us +- 0.5 us: 1.52x slower > > (+52%)> ... > > ... > > unpickle and unpickle_list shows massive slowdown. I suspect this slowdown > > i

Re: [Python-Dev] Compact ordered set

2019-02-26 Thread Victor Stinner
Le mar. 26 févr. 2019 à 12:33, INADA Naoki a écrit : > - unpickle_list: 8.48 us +- 0.09 us -> 12.8 us +- 0.5 us: 1.52x slower > (+52%)> ... > ... > unpickle and unpickle_list shows massive slowdown. I suspect this slowdown > is not caused from set change. Linux perf shows many pagefault is happ