Writing a lot of performance tests creates the problem that those tests will take a long time to run. The nature of performance tests is that each test must run for a relatively long time to get meaningful results. Therefore I doubt writing lots of different performance tests can scale. (Maybe we can find ways to eliminate noise in very short tests, but that might be research.)
One other thing to keep in mind if we're going to start doing performance tests differently is https://bugzilla.mozilla.org/show_bug.cgi?id=846166. Basically Chris suggests using eideticker for performance tests a lot more. Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28] _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
