In a message of Tue, 08 Mar 2011 18:17:17 +0100, Miquel Torres writes: >you mean this timeline, right?: >http://speed.pypy.org/timeline/?ben=3Dspectral-norm > >Because the December 22 result is so high, the yaxis maximum goes up >to 2.5, thus having less space for the more interesting < 1 range, >right?
yes > >Regarding mozilla, do you mean this site?: http://arewefastyet.com/ >I can see their timelines have some holes, probably failed runs... I was seeing something else, and I don't have a url. I think that what I was seeing is what they use to make the arewefastyet.com site. >I see a problem with the approach you suggest. Entering an arbitrary >maximum yaxis number is not a good thing. I think the onus is there on >the benchmark infrastructure to not send results that aren't >statistically significant. See Javastats >(http://www.elis.ugent.be/en/JavaStats), or ReBench >(https://github.com/smarr/ReBench). I don't think you understand what I want. Sorry if I was unclear. I am fine with the way that the benchmarks are displayed right now, but I want a way to dynamically do there and say, I want to throw away all data that is higher than a certain figure, or lower than a certain one, because right now I am onoy interested in results in a certain range. I'm not looking to change what the benchmark says for everybody who looks at it, or change how it is presented in general. I just want a way to zoom in and only see results in the range that interests me. You and anybody else might have a different range that interests you, and you should be free to get this as well. >Something that can be done on the Codespeed side is to treat >differently points that have a too high stddev. In the aforementioned >spectral-norm timeline, the stddev "floor" is around 0.0050, while the >spike has a 0.30 stddev, much higher. A "strict" mode could be >implemented that invalidates or hides statistically unsound data. The problem is that I want to throw away arbitrary amounts of data regardless of whether they are statistically significant or not, on the basis of I know what I want to see, and this other stuff is getting in the way or being distractingÃ. >Btw., I had written to the arewefastyet guys about the possibility of >configuring a Codespeed instance for them. We may yet see >collaboration there ;-) > >Miquel Laura _______________________________________________ [email protected] http://codespeak.net/mailman/listinfo/pypy-dev
