[issue32623] Resize dict on del/pop

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue32623] Resize dict on del/pop

Change by Raymond Hettinger : -- nosy: +rhettinger, tim.peters ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue32623] Resize dict on del/pop

Change by Guido van Rossum : -- nosy: +gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mai

[issue32623] Resize dict on del/pop

Change by INADA Naoki : -- dependencies: +GROWTH_RATE prevents dict shrinking ___ Python tracker ___ ___ Python-bugs-list mailing lis

[issue32623] Resize dict on del/pop

Change by Xavier G. Domingo : -- nosy: +xgdomingo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue32623] Resize dict on del/pop

INADA Naoki added the comment: I think I understand #17563, and I should fix GROWTH_RATE. Before compact-ordered dict, we can avoid resizing in "the number of deletions is on a par with the number of insertions." scenario, by large GROWTH_RATE. That's because new entry can reuse dummy entries.

[issue32623] Resize dict on del/pop

INADA Naoki added the comment: > Should we make sure that all dicts have at least MINSIZE entries? I don't think so. I think "allocate on first insert" is good idea for dict.clear() And I think it's good idea for new dict too: https://github.com/python/cpython/pull/1080 -- _

[issue32623] Resize dict on del/pop

STINNER Victor added the comment: "This will cause significant performance regression for `dict[a]=None; del dict[a]` loop. del/pop shouldn't do clear()." Should we make sure that all dicts have at least MINSIZE entries? -- ___ Python tracker

[issue32623] Resize dict on del/pop

INADA Naoki added the comment: > * When dict size become 0, make the dict shared-empty, like dict.clear() This will cause significant performance regression for `dict[a]=None; del dict[a]` loop. del/pop shouldn't do clear(). > * When (dict size < dk_size/8), call insertion_resize() This is b

[issue32623] Resize dict on del/pop

Change by INADA Naoki : -- pull_requests: +5147 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue32623] Resize dict on del/pop

Change by INADA Naoki : -- keywords: +patch pull_requests: +5144 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list

[issue32623] Resize dict on del/pop

STINNER Victor added the comment: This issue is a matter of CPU vs memory tradeoff. It reminds me the PEP 393: str type doesn't make tradeoff anymore, CPython guarantee that str always use the most efficient storage (least memory) for characters. But here the tradeoff is different. A dict is

[issue32623] Resize dict on del/pop

STINNER Victor added the comment: I agree that an heuristic is needed to decide when a dict should be compacted. > * When (dict size < dk_size/8), call insertion_resize() In bpo-31179, I suggested to Yury to use 2/3 ratio... to avoid integer overflow :-) He first used 80%, but I dislike using

[issue32623] Resize dict on del/pop

INADA Naoki added the comment: For insert/pop loop, reduce table size aggressively on pop may cause performance regression. So reducing size should be conservative. So my opinion is: * When dict size become 0, make the dict shared-empty, like dict.clear() * When (dict size < dk_size/8), call

[issue32623] Resize dict on del/pop

STINNER Victor added the comment: Note: It was recently discussed if "del dict[key]" should keep the insertion order. If I understood correctly: yes, the order must be preserved on deletion. https://mail.python.org/pipermail/python-dev/2017-November/150142.html -- ___

[issue32623] Resize dict on del/pop

New submission from Yury Selivanov : We should investigate whether we want dicts to compact themselves on del/pop operations. Currently we have to rely on workarounds to have compactable dict.copy (see issue 31179) etc. -- components: Interpreter Core messages: 310427 nosy: inada.naok