Re: disabled non-e10s tests on trunk

>you could edit the taskcluster/ci/test/tests.yml file and add >|linux64/debug: both| in the places we have |windows7-32/debug: both|, as >seen here: >http://searchfox.org/mozilla-central/source/taskcluster/ci/test/tests.yml#138 > >If there are concerns with windows build times, please ask for thos

Re: disabled non-e10s tests on trunk

be faster- it will make it better for all of us :) On Fri, Sep 8, 2017 at 5:40 PM, Ben Kelly wrote: > Joel, > > Is there an easy way for me to run non-e10s tests on linux? We often use > "t-style" try pushes where we only run tests on one platform. Restricting > non-e

Re: disabled non-e10s tests on trunk

Joel, Is there an easy way for me to run non-e10s tests on linux? We often use "t-style" try pushes where we only run tests on one platform. Restricting non-e10s to win7-debug means I either need to run tests on multiple platforms or use windows for the "t-style". I don&#

Re: disabled non-e10s tests on trunk

Yesterday I landed bug 1391371 which enabled non-e10s unittests on windows 7 debug. A few tests had to be disabled in order to do this. To keep track of what we did and each of the test suites to evaluate, I have filed bug 1391350. As a note we now have the following non-e10s tests running

Re: disabled non-e10s tests on trunk

e like mochitest to get the e10s coverage. Plus these > tests would probably take a lot execution time. > > Technically that leaves us with a somewhat blind spot for the coverage of > networking corner cases under e10s. I guess if there is a high demand for > turning off all non-

Re: disabled non-e10s tests on trunk

lot of work to create the same amount of tests with a higher level test suite like mochitest to get the e10s coverage. Plus these tests would probably take a lot execution time. Technically that leaves us with a somewhat blind spot for the coverage of networking corner cases under e10s. I guess if

Re: disabled non-e10s tests on trunk

On 16/08/17 19:36, Ben Kelly wrote: My only thought about windows7-debug is that android is a variant of linux. Running a linux platform might be closer to android behavior. But I don't have a known specific difference in mind. Right it seems like there are two use cases here: 1) Tests that

Re: disabled non-e10s tests on trunk

On Wed, Aug 16, 2017 at 2:32 PM, Joel Maher wrote: > Thanks everyone for chiming in here. I see this isn't as simple as a > binary decision and to simplify things, I think turning on all non-e10s > tests that were running for windows7-debug would give us reasonable > coverag

Re: disabled non-e10s tests on trunk

Thanks everyone for chiming in here. I see this isn't as simple as a binary decision and to simplify things, I think turning on all non-e10s tests that were running for windows7-debug would give us reasonable coverage and ensure that users on our most popular OS (and focus for 57) have a s

Re: disabled non-e10s tests on trunk

ially due to the fact that this is mostly affecting Firefox for Android where we have the lowest pre-release testing population among our tier1 products (AFAIK) makes that extremely risky. Therefore, I think we should: * start running all of the non-e10s tests that can affect Android again i

Re: disabled non-e10s tests on trunk

On 15/08/17 21:39, Ben Kelly wrote: On Tue, Aug 15, 2017 at 4:37 PM, Joel Maher wrote: All of the above mentioned tests are not run on Android (well mochitest-media is to some degree). Is 4 months unreasonable to fix the related tests that do not run in e10s? Is there another time frame that

Re: disabled non-e10s tests on trunk

On 16/08/17 01:26, Nils Ohlmeier wrote: I guess not a lot of people are aware of it, but for WebRTC we still have two distinct implementations for the networking code. So if I understand the impact here right we just lost test coverage for probably a couple of thousand lines of code. […] I’m

Re: disabled non-e10s tests on trunk

I guess not a lot of people are aware of it, but for WebRTC we still have two distinct implementations for the networking code. So if I understand the impact here right we just lost test coverage for probably a couple of thousand lines of code. And yes you can potentially blame it on WebRTC netw

Re: disabled non-e10s tests on trunk

I think Ben's argument has merit: 1. Even after Firefox 57, we will still be shipping a product in non-e10s mode: Firefox for Android 2. If WPT (and potentially other suites) aren't being run in non-e10s mode, it increases risk because we are shipping untested code paths to our users Someone migh

Re: disabled non-e10s tests on trunk

On Tue, Aug 15, 2017 at 4:43 PM, Joel Maher wrote: > This is a discussion about tests in e10s mode, not WPT on Android. > Yes. And android is our last tier 1 platform that requires non-e10s. I think that makes it relevant to the discussion. > > What specific coverage are we missing by not ru

Re: disabled non-e10s tests on trunk

This is a discussion about tests in e10s mode, not WPT on Android. What specific coverage are we missing by not running WPT in non-e10s mode on desktop? When can we ensure we have that coverage in e10s mode? I assume this is just making sure the tests that we have disabled on e10s mode need to g

Re: disabled non-e10s tests on trunk

On Tue, Aug 15, 2017 at 4:37 PM, Joel Maher wrote: > All of the above mentioned tests are not run on Android (well > mochitest-media is to some degree). Is 4 months unreasonable to fix the > related tests that do not run in e10s? Is there another time frame that > seems more reasonable? > Last

Re: disabled non-e10s tests on trunk

All of the above mentioned tests are not run on Android (well mochitest-media is to some degree). Is 4 months unreasonable to fix the related tests that do not run in e10s? Is there another time frame that seems more reasonable? On Tue, Aug 15, 2017 at 4:34 PM, Ben Kelly wrote: > On Tue, Aug 1

Re: disabled non-e10s tests on trunk

On Tue, Aug 15, 2017 at 4:29 PM, wrote: > While there are many tests which individually are disabled or lacking > coverage, these test suites have no non-e10s coverage: > * web-platform-tests > * browser-chrome > * devtools > * jsreftests > * mochitest-webgl, mochitest-gpu, mochitest-media > * re

Re: disabled non-e10s tests on trunk

Thanks everyone for commenting on this thread. As a note, we run many tests in non-e10s mode: * android mochitest, reftest, crashtest, marionette, * mochitest-chrome * xpcshell * gtest/cpptest/jittest * mochitest-a11y * mochitest-jetpack (very few tests remain) While there are many tests which i

Re: disabled non-e10s tests on trunk

We now have the ability to set prefs from a mochitest manifest (see bug 1328830 and my recent newsgroup post). We could refactor these tests into a special non-e10s manifest that sets browser.tabs.remote.autostart=false and keep running them as

Re: disabled non-e10s tests on trunk

ng reasons to run some tests in e10s based on specific coverage. >> With that said, I would like to say that any tests we turn on as non-e10s >> must have a clearly marked end date- as in this is only a temporary measure >> while we schedule work in to gain this coverage fully wi

Re: disabled non-e10s tests on trunk

I ran some scripts that I had to find out tests that are *fully* disabled on e10s, and posted the results to https://bugzilla.mozilla.org/show_bug.cgi?id=1376934 In summary: mochitest-plain: 49 tests browser-chrome: 15 tests devtools: 86 tests Note that the script evaluates the skip-if condition

Re: disabled non-e10s tests on trunk

On Wed, Aug 9, 2017 at 3:36 AM, Boris Zbarsky wrote: > > Hmm. Do we load about:blank from the url bar in a content process? > > Yes. I agree, I find it annoying too that we have to rely on MOZ_DEBUG_CHILD_PROCESS or MOZ_DEBUG_CHILD_PAUSE and that I have to be clever all the time about how to hi

Re: disabled non-e10s tests on trunk

On 8/8/17 6:40 PM, Blake Kaplan wrote: What part of the current set-up is rocket science? Debugging pageload. Especially pageload with no session history. In multi, there's definitely a problem figuring out which process is the active one (though tooltips when hovering over tabs can help).

Re: disabled non-e10s tests on trunk

On 08/08/2017 06:51 PM, Blake Kaplan wrote: L. David Baron wrote: Has there been any additional effort to deal with tests that have been disabled under e10s? This change means we've essentially turned off a decent number of tests. IIRC, the last time we talked about this, there was interest i

Re: disabled non-e10s tests on trunk

L. David Baron wrote: > Has there been any additional effort to deal with tests that have > been disabled under e10s? This change means we've essentially > turned off a decent number of tests. IIRC, the last time we talked about this, there was interest in either running tests explicitly disable

Re: disabled non-e10s tests on trunk

Boris Zbarsky wrote: > On 8/8/17 5:12 PM, jma...@mozilla.com wrote: >> In bug 1386689, we have turned them off. There was some surprise in doing >> this and some valid concerns expressed in comments in the bug. > Something as simple as "debug something that happens during pageload" is > current

Re: disabled non-e10s tests on trunk

On Wed, Aug 9, 2017 at 9:31 AM, Boris Zbarsky wrote: Something as simple as "debug something that happens during pageload" is > currently fairly rocket science to do in e10s mode; doubly so in > e10s-multi. I haven't seen any concrete proposals for improving this rr makes it fairly easy on

Re: disabled non-e10s tests on trunk

On Tuesday 2017-08-08 14:12 -0700, jma...@mozilla.com wrote: > As Firefox 57 is on trunk, we are shipping e10s by default. This means that > our primary support is for e10s. As part of this, there is little to no need > to run duplicated tests in non-e10s and e10s mode. > > In bug 1386689, w

Re: disabled non-e10s tests on trunk

On 8/8/17 5:12 PM, jma...@mozilla.com wrote: In bug 1386689, we have turned them off. There was some surprise in doing this and some valid concerns expressed in comments in the bug. Indeed. Given how often non-e10s mode needs to get used for debugging, it's a little concerning to see the "y

Re: disabled non-e10s tests on trunk

e tests in e10s based on specific coverage. >> With that said, I would like to say that any tests we turn on as non-e10s >> must have a clearly marked end date- as in this is only a temporary measure >> while we schedule work in to gain this coverage fully with e10s tests. >>

Re: disabled non-e10s tests on trunk

ts we turn on as non-e10s > must have a clearly marked end date- as in this is only a temporary measure > while we schedule work in to gain this coverage fully with e10s tests. > If we have an android test disabled (a lot I think), then we should consider continuing to run the test in non-

disabled non-e10s tests on trunk

coverage fully with e10s tests. Also keep in mind that most if not all of the CI/scheduling/admin benefits of turning off the tests are already accounted for with the new stylo tests we are running in parallel on osx and windows. ___ dev-platform

Re: e10s tests

On Tuesday, January 5, 2016 at 2:38:30 PM UTC-5, Felipe G wrote: > (cross-posted to fx-dev and dev.platform) > > As we drive towards shipping e10s, we are working on making sure that our > tests work with e10s. As you already know, there's a number of tests that > are disabled from running on e10s

Re: e10s tests

I've been a little hesitant to plug this as it's a prototype and the UX is pretty terrible. Also the underlying database it's using is not super resilient. But people might find it useful for this effort and I don't have time to improve it much more anyway, so what the heck. This is basically wha

e10s tests

(cross-posted to fx-dev and dev.platform) As we drive towards shipping e10s, we are working on making sure that our tests work with e10s. As you already know, there's a number of tests that are disabled from running on e10s, and we need to enable them. We've created a spreadsheet that lists every

Re: New e10s tests on tinderbox

- Original Message - > From: "Ehsan Akhgari" > To: "Bill McCloskey" , "dev-platform" > > Sent: Wednesday, April 9, 2014 6:51:46 AM > Subject: Re: New e10s tests on tinderbox > > Just to be explicit, this means that changesets which re

Re: New e10s tests on tinderbox

on inbound, central, try, fx-team, and b2g-inbound. In a few days, they'll be running on all trunk trees. If you do a try push, e10s tests will run iff mochitests-plain run. We don't have a specific trychooser syntax for them yet. The tests are restricted to Linux and Linux64 opt builds r

Re: New e10s tests on tinderbox

> Right now, these tests are running on inbound, central, try, fx-team, > and b2g-inbound. In a few days, they'll be running on all trunk trees. If > you do a try push, e10s tests will run iff mochitests-plain run. We don't > have a specific trychooser syntax for them yet. >

Re: New e10s tests on tinderbox

running on inbound, central, try, fx-team, and > b2g-inbound. In a few days, they'll be running on all trunk trees. If you do > a try push, e10s tests will run iff mochitests-plain run. We don't have a > specific trychooser syntax for them yet. > > The tests are restrict

Re: New e10s tests on tinderbox

Bill McCloskey wrote: > Starting today, we have new mochitests that show up as M-e10s (1 2 3 4 5). > These are mochitests-plain running inside an e10s content process. Aside from > being in a separate process, they work pretty much the same as normal. Some > tests have been disabled for e10s. I

Re: New e10s tests on tinderbox

- Original Message - > From: "Bobby Holley" > To: "Bill McCloskey" > Cc: "dev-platform" > Sent: Tuesday, April 8, 2014 2:35:26 PM > Subject: Re: New e10s tests on tinderbox > > Can you elaborate on the kinds of things that make tests

Re: New e10s tests on tinderbox

> Most of the work for this was done by Ted, Armen, Aki, and Mark Hammond. > Thanks guys! And RyanVM! I knew I'd forget someone. Sorry. -Bill ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform

Re: New e10s tests on tinderbox

This is awesome! Great job getting us this far. On Tue, Apr 8, 2014 at 2:28 PM, Bill McCloskey wrote: > We have about 85% of mochitests-plain running right now. Can you elaborate on the kinds of things that make tests fail on e10s? I have some idea in my head of what they might be, but I don't

Re: New e10s tests on tinderbox

inbound, central, try, fx-team, and > b2g-inbound. In a few days, they'll be running on all trunk trees. If you do > a try push, e10s tests will run iff mochitests-plain run. We don't have a > specific trychooser syntax for them yet. > > The tests are restricted to Linux

New e10s tests on tinderbox

bugs that I want to fix first. In the meantime, we can at least identify regressions in the tests that run. Right now, these tests are running on inbound, central, try, fx-team, and b2g-inbound. In a few days, they'll be running on all trunk trees. If you do a try push, e10s tests will run