[issue28832] Reduce memset in dict creation

Change by INADA Naoki : -- resolution: -> wont fix stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue28832] Reduce memset in dict creation

INADA Naoki added the comment: > I think that clearing 120 bytes at a time is faster than clear it later > entry-by-entry. Ah, my word was wrong. This patch skips zero clear entirely. In pseudo code: // When allocating PyDictKeyObject. - memset(dk_entries, 0, sizeof(dk_entries)); // When

[issue28832] Reduce memset in dict creation

INADA Naoki added the comment: I ran pyperformance again on more large machine (Azure D2 -> D4) for more robust result. $ ./python-default -m perf compare_to default.json.gz patched.json.gz -G Slower (4): - xml_etree_generate: 425 ms +- 18 ms -> 442 ms +- 19 ms: 1.04x slower - call_method: 25.7

[issue28832] Reduce memset in dict creation

Changes by INADA Naoki : Added file: http://bugs.python.org/file45699/default.json.gz ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue28832] Reduce memset in dict creation

STINNER Victor added the comment: You might experiment Python_Calloc(). I'm not sure that this allocator is well optimized (especially the pymalloc allocator) :-/ Last time I played with it, it was slower, especially for allocations smaller than 1 MB. -- _

[issue28832] Reduce memset in dict creation

Serhiy Storchaka added the comment: I think that clearing 120 bytes at a time is faster than clear it later entry-by-entry. Your patch removes some asserts, this looks not good. Could your provide microbenchmarks that show the largest speed up and the largest slow down? So we would see what t

[issue28832] Reduce memset in dict creation

STINNER Victor added the comment: Can you please attached the two JSON files, compressed with gzip (.gz files). perf is also to read gzipped JSON. -- ___ Python tracker ___

[issue28832] Reduce memset in dict creation

INADA Naoki added the comment: $ ./python-default -m perf compare_to default.json patched.json -G Slower (2): - xml_etree_iterparse: 328 ms +- 26 ms -> 350 ms +- 29 ms: 1.07x slower - fannkuch: 1.58 sec +- 0.09 sec -> 1.65 sec +- 0.09 sec: 1.05x slower Faster (9): - scimark_sor: 870 ms +- 59 ms

[issue28832] Reduce memset in dict creation

New submission from INADA Naoki: This patch delays initialization of dk_entries. Entries are initialized when first use (when dk_netries is incremented). Minimum dk_entries of 64bit arch is 5 * 8 * 3 = 120bytes. So it can save 4 cache lines! I'm running benchmark for now. -- assignee: