On Fri, Jun 23, 2017 at 4:19 AM, Chris Peterson <cpeter...@mozilla.com> wrote:
> > > On 6/23/17 12:17 AM, Ehsan Akhgari wrote: > > But to speak of a more direct measurement of performance, let's look at > our progress on Speedometer V2 > <https://github.com/WebKit/webkit/tree/master/PerformanceTests/Speedometer>. > Today, I measured our progress so far on this benchmark by comparing > Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest > Nightly, all x64 builds, on the reference hardware > <https://www.amazon.com/dp/B01K1IO3QW>. This is the result (numbers are > the reported benchmark score, higher is better): > > [image: Speedometer improvements] > > > How do these Speedometer V2 scores map to the results on AWFY? AWFY shows > many Speedometer sub-tests, but no score in the range of 70.21. AWFY > machine #36 is the reference hardware. > > https://arewefastyet.com/#machine=36&view=breakdown&suite=speedometer-misc > Armen has been investigating the difference. The Speedometer benchmark is extremely sensitive to anything else that is going on on the machine at the time you are running the tests, see for example https://bugzilla.mozilla.org/show_bug.cgi?id=1373396#c3 where he discovered that turning off the "Shut off display after 15 minutes" setting improves our benchmark score by about 10 points or so! I think there are other investigations ongoing to dig into the remaining difference as well. (Also note that there is a speed difference on Speedometer between Nightly and Beta, where Nightly with the same code will be a bit slower than Beta since some Nightly specific features do show up in Speedometer profiles currently, for example things like bug 1375568 are currently Nightly only, and there are also debugging checks like < https://searchfox.org/mozilla-central/rev/3291398f10dcbe192fb52e74974b172616c018aa/ipc/chromium/src/base/pickle.h#26> that show up a bit in profiles as well. I think that explains why 55.0b3 is scoring so high comparing to 56.0a1 there.) Cheers, -- Ehsan _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
