[issue22727] Improve benchmarks precision

2014-10-25 Thread Antoine Pitrou
Antoine Pitrou added the comment: You thereotically need 2**63 runs to select the best answer, though. You don't know how the distribution of hash seeds can influence the results, which means mere sampling may not be sufficient. -- ___ Python tracke

[issue22727] Improve benchmarks precision

2014-10-25 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: For more precise results it may be worth to run interpreters several times with random hash seed and then select the best time. Different interpreters can have different effect on the same seed. -- ___ Python trac

[issue22727] Improve benchmarks precision

2014-10-25 Thread Antoine Pitrou
Antoine Pitrou added the comment: Pushed in dc7d29be5a9e. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ _

[issue22727] Improve benchmarks precision

2014-10-25 Thread Georg Brandl
Georg Brandl added the comment: LGTM. -- nosy: +georg.brandl ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue22727] Improve benchmarks precision

2014-10-25 Thread Antoine Pitrou
New submission from Antoine Pitrou: This patch tries to improve precision of benchmarks in the benchmark suite by two measures: - select the best timer for the pair of interpreters (i.e. perf_counter() if possible) - make hashing deterministic to avoid fluctuations between runs -- comp