[issue24076] sum() several times slower on Python 3 64-bit

Pablo Galindo Salgado added the comment: Always happy to help :) -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubs

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: Sorry for that, Pablo. I knew exactly where the problem was, the second I read your notification. Thank you for resolving it so quickly. -- ___ Python tracker __

[issue24076] sum() several times slower on Python 3 64-bit

Pablo Galindo Salgado added the comment: New changeset 1c7e98dc258a0e7ccd2325a1aefc4aa2de51e1c5 by Pablo Galindo Salgado in branch 'main': bpo-24076: Fix reference in sum() introduced by GH-28469 (GH-28493) https://github.com/python/cpython/commit/1c7e98dc258a0e7ccd2325a1aefc4aa2de51e1c5 --

[issue24076] sum() several times slower on Python 3 64-bit

Pablo Galindo Salgado added the comment: Sorry, I meant PR 28493 -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubs

[issue24076] sum() several times slower on Python 3 64-bit

Pablo Galindo Salgado added the comment: Opened #28493 to fix the refleak -- ___ Python tracker ___ ___ Python-bugs-list mailing li

[issue24076] sum() several times slower on Python 3 64-bit

Change by Pablo Galindo Salgado : -- pull_requests: +26888 pull_request: https://github.com/python/cpython/pull/28493 ___ Python tracker ___ ___

[issue24076] sum() several times slower on Python 3 64-bit

Pablo Galindo Salgado added the comment: Unfortunately commit debd80403721b00423680328d6adf160a28fbff4 introduced a reference leak: ❯ ./python -m test test_grammar -R : 0:00:00 load avg: 2.96 Run tests sequentially 0:00:00 load avg: 2.96 [1/1] test_grammar beginning 9 repetitions 123456789 ..

[issue24076] sum() several times slower on Python 3 64-bit

Serhiy Storchaka added the comment: Thank you again Stefan. Now no doubts are left. BTW, pyperf gives more stable results. I use it if have any doubts (either the results of timeit are not stable or the difference is less than say 10%). -- ___ Pyt

[issue24076] sum() several times slower on Python 3 64-bit

Change by Stefan Behnel : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: Old, with PGO: $ ./python -m timeit -s 'd = list(range(2**61, 2**61 + 1))' 'sum(d)' 1000 loops, best of 5: 340 usec per loop $ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)' 2000 loops, best of 5: 114 usec per loop $ ./python -m timei

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: Hmm, thanks for insisting, Serhiy. I was accidentally using a debug build this time. I'll make a PGO build and rerun the microbenchmarks. -- ___ Python tracker _

[issue24076] sum() several times slower on Python 3 64-bit

Serhiy Storchaka added the comment: Thank you. Could you please test PGO builds? -- ___ Python tracker ___ ___ Python-bugs-list mai

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: Original: $ ./python -m timeit -s 'd = list(range(2**61, 2**61 + 1))' 'sum(d)' 500 loops, best of 5: 712 usec per loop $ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)' 2000 loops, best of 5: 149 usec per loop $ ./python -m timeit -s '

[issue24076] sum() several times slower on Python 3 64-bit

Change by Stefan Behnel : -- Removed message: https://bugs.python.org/msg402301 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: Original: $ ./python -m timeit -s 'd = list(range(2**61, 2**61 + 1))' 'sum(d)' 500 loops, best of 5: 712 usec per loop $ ./python -m timeit -s 'd = list(range(2**30, 2**30 + 1))' 'sum(d)' 2000 loops, best of 5: 149 usec per loop $ ./python -m timeit -s '

[issue24076] sum() several times slower on Python 3 64-bit

Serhiy Storchaka added the comment: What are microbenchmark results for PR 28469 in comparison with the baseline? -- ___ Python tracker ___ ___

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: New changeset debd80403721b00423680328d6adf160a28fbff4 by scoder in branch 'main': bpo-24076: Inline single digit unpacking in the integer fastpath of sum() (GH-28469) https://github.com/python/cpython/commit/debd80403721b00423680328d6adf160a28fbff4 ---

[issue24076] sum() several times slower on Python 3 64-bit

Guido van Rossum added the comment: Sounds good, you have my blessing. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: > The patch looks fine, but it looks a bit like benchmark chasing. Is the speed > of builtin sum() of a sequence of integers important enough to do this bit of > inlining? Given that we already accepted essentially separate loops for the int, float and every

[issue24076] sum() several times slower on Python 3 64-bit

Guido van Rossum added the comment: The patch looks fine, but it looks a bit like benchmark chasing. Is the speed of builtin sum() of a sequence of integers important enough to do this bit of inlining? (It may break if we change the internals of Py_Long, as Mark Shannon has been wanting to d

[issue24076] sum() several times slower on Python 3 64-bit

Raymond Hettinger added the comment: > I created a PR from my last patch, inlining the unpacking > of single digit integers. Thanks, that gets to the heart of the issue. I marked the PR as approved (though there is a small coding nit you may want to fix). -- _

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: I created a PR from my last patch, inlining the unpacking of single digit integers. Since most integers should fit into a single digit these days, this is as fast a path as it gets. https://github.com/python/cpython/pull/28469 -- versions: +Python 3.

[issue24076] sum() several times slower on Python 3 64-bit

Change by Stefan Behnel : -- pull_requests: +26868 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/28469 ___ Python tracker ___

[issue24076] sum() several times slower on Python 3 64-bit

Raymond Hettinger added the comment: > OTOH on my Mac I still find that 3.10 with PGO is still > more than twice as slow than 2.7. > Thinking about it that's a bit odd, since (presumably) > the majority of the work in sum() involves a long int result > (even though the values returned by ra

[issue24076] sum() several times slower on Python 3 64-bit

Guido van Rossum added the comment: @Stefan > FWIW, a PGO build of Py3.7 is now about 20% *faster* here than my Ubuntu > 16/04 system Python 2.7 Does that mean we can close this issue? Or do I misunderstand what you are comparing? 32 vs. 64 bits? PGO vs. non-PGO? OTOH on my Mac I still find

[issue24076] sum() several times slower on Python 3 64-bit

Change by Antoine Pitrou : -- nosy: -pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyt

[issue24076] sum() several times slower on Python 3 64-bit

Stefan Behnel added the comment: FWIW, a PGO build of Py3.7 is now about 20% *faster* here than my Ubuntu 16/04 system Python 2.7, and for some (probably unrelated) reason, the system Python 3.5 is another 2% faster on my side. IMHO, the only other thing that seems obvious to try would be to

[issue24076] sum() several times slower on Python 3 64-bit

Changes by Louie Lu : -- title: sum() several times slower on Python 3 -> sum() several times slower on Python 3 64-bit ___ Python tracker ___ __

[issue24076] sum() several times slower on Python 3

Changes by Louie Lu : -- versions: +Python 3.7 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyt

[issue24076] sum() several times slower on Python 3

Stefan Behnel added the comment: Issue 24165 was created to pursue the path of a free-list for PyLong objects. -- ___ Python tracker ___ _

[issue24076] sum() several times slower on Python 3

Steven D'Aprano added the comment: Antoine asked: > If someone has actual code that suffers from this, it would be good to know > about it. You might have missed Łukasz' earlier comment: "In this particular case, there's internal usage at Twitter that unearthed it. The example is just a simp

[issue24076] sum() several times slower on Python 3

Stefan Behnel added the comment: I tried implementing a freelist. Patch attached, mostly adapted from the one in dictobject.c, but certainly needs a bit of cleanup. The results are not bad, about 10-20% faster: Original: $ ./python -m timeit 'sum(range(1, 10))' 1000 loops, best of 3: 1.86

[issue24076] sum() several times slower on Python 3

Antoine Pitrou added the comment: Le 01/05/2015 08:09, Stefan Behnel a écrit : > > I don't think it's irrelevant. Throw-away integers are really not uncommon. For-loops use them quite often, non-trivial arithmetic expressions can create a lot of intermediate temporaries. Speeding up the create-d

[issue24076] sum() several times slower on Python 3

Stefan Behnel added the comment: I don't think it's irrelevant. Throw-away integers are really not uncommon. For-loops use them quite often, non-trivial arithmetic expressions can create a lot of intermediate temporaries. Speeding up the create-delete cycle of PyLong sounds like a very obvious

[issue24076] sum() several times slower on Python 3

Antoine Pitrou added the comment: It seems we (like the benchmarks posted) are spending a whole lot of time on something that's probably not relevant to any real-world situation. If someone has actual code that suffers from this, it would be good to know about it. (note by the way that summing

[issue24076] sum() several times slower on Python 3

Stefan Behnel added the comment: > there are three ingredients here - sum, (x)range and the integer addition > that sum will be performing at each iteration. ... not to forget the interpreter startup time on his machine. :) I did a tiny bit of profiling and about 90% of the time seems to be sp

[issue24076] sum() several times slower on Python 3

Mark Dickinson added the comment: Throwing out sum, I'm seeing significant slowdown simply from xrange versus range: taniyama:Desktop mdickinson$ python2 -m timeit -s 'x = xrange(3, 10**9, 3)' 'for e in x: pass' 10 loops, best of 3: 5.01 sec per loop taniyama:Desktop mdickinson$ python3 -m tim

[issue24076] sum() several times slower on Python 3

Mark Dickinson added the comment: Łukasz: there are three ingredients here - sum, (x)range and the integer addition that sum will be performing at each iteration. Is there any chance you can separate the effects on your machine? On my machine (OS X, 64-bit), I'm seeing *some* speed difference

[issue24076] sum() several times slower on Python 3

Antoine Pitrou added the comment: If that's due to the different representation of Python 2's int type and Python 3's int type then I don't see an easy solution to this. -- versions: -Python 3.4 ___ Python tracker

[issue24076] sum() several times slower on Python 3

Łukasz Langa added the comment: Serhiy, this is 64-bit specific. Antoine, as far as I can tell, the main use case is: "Don't make it look like migrating to Python 3 is a terrible performance downgrade." As we discussed on the language summit this year [1], we have to be at least not worse to

[issue24076] sum() several times slower on Python 3

Antoine Pitrou added the comment: I reproduce under 64-bit Linux. So this may be because the Python long digit (30 bits) is smaller than the C long (64 bits). Lukasz: is there a specific use case? Note you can use Numpy for such calculations. -- __

[issue24076] sum() several times slower on Python 3

Serhiy Storchaka added the comment: Can't reproduce on 32-bit Linux. $ time python2.7 -c "print sum(xrange(3, 10**9, 3)) + sum(xrange(5, 10**9, 5)) - sum(xrange(15, 10**9, 15))" 216668 real1m11.614s user1m11.376s sys 0m0.056s $ time python3.4 -c "print(sum(range(3, 10**

[issue24076] sum() several times slower on Python 3

New submission from Łukasz Langa: I got a report that summing numbers is noticably slower on Python 3. This is easily reproducible: $ time python2.7 -c "print sum(xrange(3, 10**9, 3)) + sum(xrange(5, 10**9, 5)) - sum(xrange(15, 10**9, 15))" 216668 real0m6.165s user0m6.100