On 12-08-15 6:10 PM, Gregory Szorc wrote:
On 8/15/12 2:52 AM, Aryeh Gregor wrote:
On Tue, Aug 14, 2012 at 10:47 PM, Gregory Szorc <g...@mozilla.com> wrote:
Is there a tracking bug for areas where we could gain efficiency? We all
know the build phase is full of clownshoes. But, I believe we also do
silly
things like execute some tests serially, only taking advantage of 1/N
CPU
cores in the process. This is just wasting resources. See [1] for a
concrete
example.
Don't we execute *all* tests serially?
Outside of splitting some suites into chunks and running on different
builders, I'm pretty sure we do.
Yes, that's true.
Many of our tests require
focus, so you can't do two runs in parallel on the same desktop. In
theory we could specially flag the ones that don't need focus, and
make sure to always run them without focus -- that would probably be
most of the tests. Then those could be run in parallel. They could
also be run in the background on developer machines, which would be
nice. This would require a bunch of developer work.
There are test suites like xpcshell and js reftests that AFAIK don't
have special hardware constraints (like the visual tests do). They can
at least be executed with full process + profile isolation. In some
cases, we could probably re-use processes to avoid that overhead. I view
each test suite as independent and I'm sure there are some low-hanging
fruits in there.
There are xpcshell tests which do things like file system operations
which might race against other tests if we run them in parallel. Of
course that's something that could be fixed, but it is not necessarily
low hanging fruit (unless someone tries this and proves me wrong, which
I will be happy to see!)
Tests in the rest of our framework might implicitly depend on the
execution environment left behind by the previous tests. This is bad
practice for sure, but it's the fact of the matter. Which is why the
focus issue is not the only thing that needs to be fixed if we're
looking at parallelizing other test runs.
That all being said, it would be *fantastic* if we can pull this off at
some point. It just requires engineering and releng resources.
Cheers,
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
