Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Stephen J. Turnbull
Raymond Hettinger writes: > We're trying to keep performant the ones that people actually use. > For the Mac, I think there are only four that matter: > > 1) The one we distribute on the python.org > website at > https://www.python.org/ftp/python/3.8.0/python-3.8.0a2-macosx10.9.pkg >

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Raymond Hettinger
On Feb 26, 2019, at 2:28 PM, Neil Schemenauer wrote: > > Are you compiling with --enable-optimizations (i.e. PGO)? In my > experience, that is needed to get meaningful results. I'm not and I would worry that PGO would give less stable comparisons because it is highly sensitive to changes its t

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Victor Stinner
Le mer. 27 févr. 2019 à 00:17, Victor Stinner a écrit : > My sad story with code placement: > https://vstinner.github.io/analysis-python-performance-issue.html > > tl; dr Use PGO. Hum wait, this article isn't complete. You have to see the follow-up: https://bugs.python.org/issue28618#msg286662 "

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Victor Stinner
Hi, PGO compilation is very slow. I tried very hard to avoid it. I started to annotate the C code with various GCC attributes like "inline", "always_inline", "hot", etc.. I also experimented likely/unlikely Linux macros which use __builtin_expect(). At the end... my efforts were worthless. I stil

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Neil Schemenauer
On 2019-02-26, Raymond Hettinger wrote: > That said, I'm only observing the effect when building with the > Mac default Clang (Apple LLVM version 10.0.0 (clang-1000.11.45.5). > When building GCC 8.3.0, there is no change in performance. My guess is that the code in _PyEval_EvalFrameDefault() got c

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Victor Stinner
Le mar. 26 févr. 2019 à 22:45, Raymond Hettinger a écrit : > Victor said he generally doesn't care about 5% regressions. That makes sense > for odd corners of Python. The reason I was concerned about this one is that > it hits the eval-loop and seems to effect every single op code. The > reg

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Raymond Hettinger
On Feb 25, 2019, at 8:23 PM, Eric Snow wrote: > > So it looks like commit ef4ac967 is not responsible for a performance > regression. I did narrow it down to that commit and I can consistently reproduce the timing differences. That said, I'm only observing the effect when building with the M

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Victor Stinner
I made an attempt once and it was faster: https://faster-cpython.readthedocs.io/registervm.html But I had bugs and I didn't know how to implement correctly a compiler. Victor Le mardi 26 février 2019, Neil Schemenauer a écrit : > On 2019-02-25, Eric Snow wrote: >> So it looks like commit ef4ac9

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Neil Schemenauer
On 2019-02-25, Eric Snow wrote: > So it looks like commit ef4ac967 is not responsible for a performance > regression. I did a bit of exploration myself and that was my conclusion as well. Perhaps others would be interested in how to use "perf" so I did a little write up: https://discuss.python.o

Re: [Python-Dev] Possible performance regression

2019-02-26 Thread Victor Stinner
Hi, Le mar. 26 févr. 2019 à 05:27, Eric Snow a écrit : > I ran the "performance" suite (https://github.com/python/performance), > which has 57 different benchmarks. Ah yes, by the way: I also ran manually performance on speed.python.org yesterday: it added a new dot at Feb 25. > In the results,

Re: [Python-Dev] Possible performance regression

2019-02-25 Thread Eric Snow
On Mon, Feb 25, 2019 at 10:42 AM Eric Snow wrote: > I'll look into it around then too. See https://bugs.python.org/issue33608. I ran the "performance" suite (https://github.com/python/performance), which has 57 different benchmarks. In the results, 9 were marked as "significantly" different bet

Re: [Python-Dev] Possible performance regression

2019-02-25 Thread Eric Snow
On Mon, Feb 25, 2019 at 10:32 AM Raymond Hettinger wrote: > I got it down to two checkins before running out of time: > > Between > git checkout 463572c8beb59fd9d6850440af48a5c5f4c0c0c9 > > And: > git checkout 3b0abb019662e42070f1d6f7e74440afb1808f03 > > So the subinterpreter patch was lik

Re: [Python-Dev] Possible performance regression

2019-02-25 Thread Raymond Hettinger
> On Feb 25, 2019, at 2:54 AM, Antoine Pitrou wrote: > > Have you tried bisecting to find out the offending changeset, if there > any? I got it down to two checkins before running out of time: Between git checkout 463572c8beb59fd9d6850440af48a5c5f4c0c0c9 And: git checkout 3b0abb019

Re: [Python-Dev] Possible performance regression

2019-02-25 Thread Antoine Pitrou
On Sun, 24 Feb 2019 20:54:02 -0800 Raymond Hettinger wrote: > I'll been running benchmarks that have been stable for a while. But between > today and yesterday, there has been an almost across the board performance > regression. Have you tried bisecting to find out the offending changeset, i

Re: [Python-Dev] Possible performance regression

2019-02-25 Thread Victor Stinner
Hi, Le lun. 25 févr. 2019 à 05:57, Raymond Hettinger a écrit : > I'll been running benchmarks that have been stable for a while. But between > today and yesterday, there has been an almost across the board performance > regression. How do you run your benchmarks? If you use Linux, are you usi

Re: [Python-Dev] Possible performance regression

2019-02-25 Thread Raymond Hettinger
> On Feb 24, 2019, at 10:06 PM, Eric Snow wrote: > > I'll look into it in more depth tomorrow. FWIW, I have a few commits > in the range you described, so I want to make sure I didn't slow > things down for us. :) Thanks for looking into it. FWIW, I can consistently reproduce the results sev

Re: [Python-Dev] Possible performance regression

2019-02-24 Thread Eric Snow
On Sun, Feb 24, 2019 at 10:04 PM Eric Snow wrote: > I'll take a look tonight. I made 2 successive runs of the script (on my laptop) for a commit from early Saturday, and 2 runs from a commit this afternoon (close to master). The output is below, with the earlier commit first. That one is a litt

Re: [Python-Dev] Possible performance regression

2019-02-24 Thread Eric Snow
I'll take a look tonight. -eric On Sun, Feb 24, 2019, 21:54 Raymond Hettinger wrote: > I'll been running benchmarks that have been stable for a while. But > between today and yesterday, there has been an almost across the board > performance regression. > > It's possible that this is a measure

[Python-Dev] Possible performance regression

2019-02-24 Thread Raymond Hettinger
I'll been running benchmarks that have been stable for a while. But between today and yesterday, there has been an almost across the board performance regression. It's possible that this is a measurement error or something unique to my system (my Mac installed the 10.14.3 release today), so