Reply inline. On 14-06-12 08:03 PM, Gregory Szorc wrote: > On 6/12/14, 2:07 PM, Armen Zambrano G. wrote: >> Hello all, >> I've recently been working on adding disabled tests into our tests.zip. >> >> There are some challenges that I have faced and I wonder what I ideas >> others have to improve it. >> >> (I'm explicitly not going to go into a blue sky scenario where tests.zip >> are not needed to begin with. If you prefer we can fork that thread as >> it would specifically help the releng integration systems: less overhead >> & network usage) >> >> == Local changes versus source code changes == >> For local testing, I make changes on the files out of the tests.zip and >> then remember to put it back into my source tree. >> >> If I'm not careful, the changes can easily be lost or never make it back >> into the source tree. >> >> == Faster tests.zip generation == >> Is there a faster way to generate a tests.zip than ./mach build and make >> package-tests from the objdir? >> >> For instance, generating a tests.zip for b2g emulators requires going >> through the whole setup of making b2g emulator builds (I'm still >> building atm). >> >> In my case, I cheated the system by downloading a tests.zip file to >> people.mozilla.com, hack it, zip it back and trigger an arbitrary job. >> >> If I start from a tests.zip and work from there (unless I need to create >> new binaries) I think I'm OK. >> >> I think this would help me, a .mach command that would: >> * download a zip file for a platform >> * unzip it >> * put my changes from the source code into it >> * zip it back >> >> What do people think? >> >> == Multiple platform == >> I've noticed that from platform to platform the tests.zip are slightly >> different. They can contain these: >> * binaries >> * runners >> * tests >> * manifests >> * anything else? >> >> In my development, it would have been awesome to generate a tests.zip >> files that would be able to run on any platform. Unfortunately, the >> binaries are coupled and test packaging is tied to the host packing >> operating system. > > (Adding dev-builds) > > We've talked about improving test archive generation from build system > land. The current solution is a giant hack and inefficient. Ted has some > ideas and a bug on file (though I can't find it ATM). We should be able > to get to a state that doesn't require a build to generate a tests archive. > > Given our new ability to trigger test jobs against arbitrary test > archives, I think having the ability to create one-off test archives for > testing is a terrific idea. I also support the idea of unified tests > archives. Although, that may require changing our moz.build parsing > strategy to include parsing every single file, despite the current build > configuration. We've considered that before, so it's not a crazy idea. > > Your ideas are terrific. I just don't know if the resources are there to > prioritize this.
Thank you! I understand about the resources. I don't see it as high priority. > > As a shim, we could probably provide a mach command to package tests and > optionally overlay in-tree test files (at least the one moz.build knows > about - test harness files are currently notably absent) over an > existing tests archive. _______________________________________________ dev-builds mailing list dev-builds@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-builds
