[issue26314] interned strings are stored in a dict, a set would use less memory

Raymond Hettinger added the comment: GPS: priority low, feel free to drop if it you want. SS: But for now status quo is good to me. Marking as closed. Feel free to open a separate tracker item if you want to pursue the use of Modules/hashtable.c. -- resolution: -> rejected status: op

[issue26314] interned strings are stored in a dict, a set would use less memory

STINNER Victor added the comment: Serhiy Storchaka added the comment: > > Since interned strings table can only grow and contains exact strings, > other data structure may be more appropriate (for example > Modules/hashtable.c). > FYI this module is not well optimized. I took code and then adapt

[issue26314] interned strings are stored in a dict, a set would use less memory

Serhiy Storchaka added the comment: Since interned strings table can only grow and contains exact strings, other data structure may be more appropriate (for example Modules/hashtable.c). But for now status quo is good to me. -- ___ Python tracker <

[issue26314] interned strings are stored in a dict, a set would use less memory

Gregory P. Smith added the comment: The space for the strings is a fixed cost, the structure used to store them for efficient lookup is the only overhead that can be trimmed and is all in one contiguous allocation. regardless, i agree, this isn't a large savings. priority low, feel free to dr

[issue26314] interned strings are stored in a dict, a set would use less memory

Raymond Hettinger added the comment: I agree with Serhiy that this should not be done. IIRC, there were some discussions on python-dev or python-ideas about using sets for interning and the judgment was the dicts are conceptually the right type and that using sets would be hack. The other t

[issue26314] interned strings are stored in a dict, a set would use less memory

Serhiy Storchaka added the comment: Dicts are hard optimized for string keys due to using them for globals and attributes lookup. Sets are optimized too, but rather for accident. We can be sure that dicts will be always optimized, but set optimization can be eliminated for the sake of simplifi

[issue26314] interned strings are stored in a dict, a set would use less memory

Gregory P. Smith added the comment: updated patch: Don't Py_CLEAR the dummy sentinel value in PySet_Fini. The interned set uses it. Otherwise it causes problems when re-initializing after a Fini as happens in processes the setup and tear down an embedded interpreter multiple times. -

[issue26314] interned strings are stored in a dict, a set would use less memory

Gregory P. Smith added the comment: Because it was only called from within an "#ifdef __INSURE__" which we weren't using. I called it an "example" patch for a reason. Updating that function to deal with the set instead of dict seems wise. Ironically... a few days after we did this we may hav

[issue26314] interned strings are stored in a dict, a set would use less memory

STINNER Victor added the comment: Why did you remove _Py_ReleaseInternedStrings()? It looks like a big memory leak when Python is embedded. -- nosy: +haypo ___ Python tracker __

[issue26314] interned strings are stored in a dict, a set would use less memory

Gregory P. Smith added the comment: Here's an example patch against 2.7 by nnorwitz that we're currently testing. -- keywords: +needs review, patch Added file: http://bugs.python.org/file41863/interned_set_27.diff ___ Python tracker

[issue26314] interned strings are stored in a dict, a set would use less memory

Josh Rosenberg added the comment: Presumably it would involve using private set APIs to make this work, right? Since normally you can't look up the actual value in a set, just check for existence? -- nosy: +josh.r ___ Python tracker

[issue26314] interned strings are stored in a dict, a set would use less memory

Changes by Maciej Szulik : -- nosy: +maciej.szulik ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue26314] interned strings are stored in a dict, a set would use less memory

Changes by Serhiy Storchaka : -- nosy: +rhettinger ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue26314] interned strings are stored in a dict, a set would use less memory

New submission from Gregory P. Smith: The implementation of string interning uses a dict [1]. It would consume less memory and be a bit simpler if it used a set. Identifier strings in a program are interned. If you have a large program with a lot of code, this makes for a large dictionary.