On Thu, Sep 15, 2016 at 5:57 PM Victor Stinner <victor.stin...@gmail.com> wrote:
> 2016-09-15 10:02 GMT+02:00 INADA Naoki <songofaca...@gmail.com>: > > In my environ: > > > > ~/local/python-master/bin/python3 -m timeit -s "d = > > dict.fromkeys(range(10**6))" 'list(d)' > > Stooooop! Please stop using timeit, it's lying! > > * You must not use the minimum but average or median > * You must run a microbenchmark in multiple processes to test > different randomized hash functions and different memory layouts > > In short: you should use my perf module. > http://perf.readthedocs.io/en/latest/cli.html#timeit > > I'm sorry. Changing habit is bit difficult. I'll use it in next time. I ran microbench 3~5 times and confirm the result is stable before posting result. And when difference is smaller than 10%, I don't believe the result. > The memory layout and the hash function have a major important on such > microbenchmark: > https://haypo.github.io/journey-to-stable-benchmark-average.html > > In this microbench, hash randomization is not important, because key of dict is int. (It means iterating dict doesn't cause random memory access in old dict implementation too.) > > > Both Python is built without neither `--with-optimizations` or `make > > profile-opt`. > > That's bad :-) For most reliable benchmarks, it's better to use > LTO+PGO compilation. > LTO+PGO may make performance of `git pull && make` unstable. PGO clean build takes tooo long time for such a quick benchmark. So I don't want to use PGO in such a quick benchmark. And Python doesn't provide way to use LTO without PGO....
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
