Re: Test helper ok() will now fail if used with more than 2 arguments

2018-11-01 Thread Ehsan Akhgari
Thank you for doing this! I cannot overstate how many times this has tripped me and others over in the past. Cheers, Ehsan On Thu, Nov 1, 2018 at 8:02 AM Julian Descottes wrote: > We are about to land https://bugzilla.mozilla.org/show_bug.cgi?id=1467712 > which changes the behavior of `ok()` t

Re: test-verify (and test-verify-wpt) now running as tier 2

2017-12-07 Thread gbrown
On Monday, October 2, 2017 at 11:12:03 AM UTC-6, Geoffrey Brown wrote: > Today the test-verify test task will start running as a tier 2 job. > Look for the "TV" symbol on treeherder, on linux-64 test platforms. > > TV is intended as an "early warning system" for identifying the > introduction of i

Re: test-verify now running as tier 2

2017-10-02 Thread Chris Peterson
This is very cool, Geoff! People have been talking about this idea for a long, so it is great to see it actually running. I'm glad to see chaos mode being tested, too. On 2017-10-02 10:11 AM, Geoffrey Brown wrote: Today the test-verify test task will start running as a tier 2 job. Look for th

Re: test

2017-02-09 Thread Xidorn Quan
Sorry for the noisy... It was meant to test whether the mailing list works again... But I picked the wrong email address so it got published. Please ignore this one. - Xidorn On Fri, Feb 10, 2017, at 01:44 PM, Xidorn Quan wrote: > This is for testing purpose, and should not be published to the li

Re: Test automation addons and signing

2016-03-21 Thread Martijn
So the xpinstall.signatures.required=false pref in about:config trick doesn't work anymore for trunki? Regards, Martijn On Tue, Mar 1, 2016 at 5:48 AM, Philip Chee wrote: > On 28/02/2016 09:45, Bobby Holley wrote: > > >> We were promised that trunk and aurora builds would not enforce addon > >>

Re: Test automation addons and signing

2016-02-29 Thread Philip Chee
On 28/02/2016 09:45, Bobby Holley wrote: >> We were promised that trunk and aurora builds would not enforce addon >> signing when xpinstall.signatures.required=false. I did not see any >> discussion about rescinding this commitment. > We also run automation on release-channel builds to test the b

Re: Test automation addons and signing

2016-02-29 Thread Jonathan Griffin
The decision not to enforce addon signing on trunk/aurora is not changing. But, to support running our automation with unsigned addons on trunk/aurora, but signed addons on beta/release, we would have to implement some pretty complex logic at the aurora -> beta uplift, and this is substantial enoug

Re: Test automation addons and signing

2016-02-27 Thread Bobby Holley
On Sat, Feb 27, 2016 at 4:33 AM, Philip Chee wrote: > On 26/02/2016 22:45, Andrew Halberstadt wrote: > > To date, our continuous integration has been setting > > 'xpinstall.signatures.required=false' to bypass addon signing. But > > soon, this pref will become obsolete and Firefox will enforce si

Re: Test automation addons and signing

2016-02-27 Thread Philip Chee
On 26/02/2016 22:45, Andrew Halberstadt wrote: > To date, our continuous integration has been setting > 'xpinstall.signatures.required=false' to bypass addon signing. But > soon, this pref will become obsolete and Firefox will enforce signing > no matter what. > > In preparation, we will begin si

Re: Test Informant Report - Week ending Dec 28

2015-01-26 Thread Andrew Halberstadt
Yeah, you'll notice there's a week missing in between there :). That would be this report: http://brasstacks.mozilla.com/testreports/weekly/2014-12-19.informant-report.html Which you'll also notice is missing a lot of data (thus why I didn't post it to dev.platform). Basically I had a fd that w

Re: Test Informant Report - Week ending Jan 25

2015-01-26 Thread Andrew Halberstadt
Yes, that's correct, it includes both enabled/added or disabled/removed. I also agree that splitting these up would be useful. Should be doable, I'll file a bug. -Andrew On 26/01/15 10:40 AM, Nathan Froyd wrote: On Mon, Jan 26, 2015 at 9:41 AM, Test Informant wrote: --- marionette

Re: Test Informant Report - Week ending Dec 28

2015-01-26 Thread Gijs Kruitbosch
I was looking at these now... it looks like the mochitest-bc-e10s count went up from 48% in the week ending Dec. 12 to 57% in the week ending Dec. 28. Yet this report records no change since the previous run. What gives? :-) ~ Gijs On 29/12/2014 15:50, Test Informant wrote: Test Informant re

Re: Test Informant Report - Week ending Jan 25

2015-01-26 Thread Nathan Froyd
On Mon, Jan 26, 2015 at 9:41 AM, Test Informant wrote: > --- > marionette- ↑0↓0 - 92% > mochitest-a11y- ↑0↓0 - 99% > mochitest-browser-chrome - ↑56↓12 - 95% > mochitest-browser-chrome-e10s - ↑24↓4 - 58% > mochitest-chrome - ↑16↓0 - 9

Re: Test Informant Report - Week ending Jan 04

2015-01-05 Thread Andrew Halberstadt
Note: Test Informant stopped collecting Android data for a bit when their builder names were changed to the new android-api-9 (etc..) format. Test Informant has now been updated. This explains the jump from 90% enabled last week down to 85% this week. -Andrew On 05/01/15 09:35 AM, Test Inform

Re: Test coverage?

2014-11-25 Thread Andrew Halberstadt
Sorry the service fell over on the weekend so I'm missing data for Sat-Mon. I just posted a report up to and including last Friday though: http://brasstacks.mozilla.com/testreports/weekly/2014-11-21.informant-report.html Daily reports should be uploaded again starting tomorrow: http://brasstacks

Re: Test coverage?

2014-11-25 Thread Gijs Kruitbosch
On 25/11/2014 10:40, Kan-Ru Chen (陳侃如) wrote: Hi, Currently we have many tests that are skipped for various reasons. Do we have data on which test runs on which platforms? For example if a test is accidentally skipped on all platforms, could we identify it? Kanru A tool called "Test Informan

Re: Test Informant Report - Week of 2014-10-05

2014-10-14 Thread Andrew Halberstadt
On 14/10/14 11:58 AM, Bill McCloskey wrote: On Tue, Oct 14, 2014 at 8:45 AM, Dave Townsend wrote: Is there any way to see the breakdown of which testsgot enabled. I'm surprised to see mochitest-plain rise by 3456 yet mochitest-plain-e10s rise by only 28. I'd love to find out which tests got tur

Re: Test Informant Report - Week of 2014-10-05

2014-10-14 Thread Bill McCloskey
On Tue, Oct 14, 2014 at 8:45 AM, Dave Townsend wrote: > Is there any way to see the breakdown of which testsgot enabled. I'm > surprised to see mochitest-plain rise by 3456 yet mochitest-plain-e10s rise > by only 28. I'd love to find out which tests got turned on on one but not > the other. If yo

Re: Test Informant Report - Week of 2014-10-05

2014-10-14 Thread Dave Townsend
Is there any way to see the breakdown of which testsgot enabled. I'm surprised to see mochitest-plain rise by 3456 yet mochitest-plain-e10s rise by only 28. I'd love to find out which tests got turned on on one but not the other. Dave On Tue, Oct 14, 2014 at 6:57 AM, Andrew Halberstadt < ahalbers

Re: Test failures in mochitest-browser caused by doing work inside DOM event handlers

2014-06-26 Thread Chris Mills
On 25 Jun 2014, at 15:18, Ed Morley wrote: > On 25/06/2014 15:16:04, Chris Mills wrote: >> It looks like a good place to put this information would be as a subsection >> of >> https://developer.mozilla.org/en/docs/Mochitest#Writing_tests >> >> Can you add in a brief section covering this point

Re: Test failures in mochitest-browser caused by doing work inside DOM event handlers

2014-06-25 Thread Ed Morley
On 25/06/2014 15:16:04, Chris Mills wrote: It looks like a good place to put this information would be as a subsection of https://developer.mozilla.org/en/docs/Mochitest#Writing_tests Can you add in a brief section covering this point, along with a brief code snippet illustrating a good and a b

Re: Test failures in mochitest-browser caused by doing work inside DOM event handlers

2014-06-25 Thread Chris Mills
On 8 May 2014, at 22:02, Irving Reid wrote: > I've recently fought my way through a bunch of intermittent test > failures in the Add-on Manager mochitest-browser suite, and there's a > common anti-pattern where tests receive a Window callback, usually > "unload", and proceed to do significant wo

Re: Test harness options now defined in-tree -- please pull m-c before pushing to try!

2014-03-26 Thread Andrew Halberstadt
On 26/03/14 01:45 PM, Nathan Froyd wrote: - Original Message - Releng (and I tend to agree with them) is very opposed to having one config affect multiple platforms (or at least multiple buildapps). Over time it tends to lead to messy and hard to follow configs. It also makes it harder i

Re: Test harness options now defined in-tree -- please pull m-c before pushing to try!

2014-03-26 Thread Nathan Froyd
- Original Message - > Releng (and I tend to agree with them) is very opposed to having one > config affect multiple platforms (or at least multiple buildapps). Over > time it tends to lead to messy and hard to follow configs. It also makes > it harder in the case that you *do* only want to

Re: Test harness options now defined in-tree -- please pull m-c before pushing to try!

2014-03-26 Thread Armen Zambrano G.
On 14-03-26 09:11 AM, Andrew Halberstadt wrote: > On 26/03/14 09:06 AM, Andrew Halberstadt wrote: >> On 26/03/14 12:19 AM, Gregory Szorc wrote: >>> Andrew: I'm curious if you or anyone else has given any consideration >>> into making it dead simple to change the config so only a subset of >>> tests

Re: Test harness options now defined in-tree -- please pull m-c before pushing to try!

2014-03-26 Thread Andrew Halberstadt
On 26/03/14 09:06 AM, Andrew Halberstadt wrote: On 26/03/14 12:19 AM, Gregory Szorc wrote: Andrew: I'm curious if you or anyone else has given any consideration into making it dead simple to change the config so only a subset of tests will be active. e.g. I'd like to save automation resources an

Re: Test harness options now defined in-tree -- please pull m-c before pushing to try!

2014-03-26 Thread Andrew Halberstadt
On 26/03/14 12:19 AM, Gregory Szorc wrote: Andrew: I'm curious if you or anyone else has given any consideration into making it dead simple to change the config so only a subset of tests will be active. e.g. I'd like to save automation resources and filter my browser chrome jobs to tests under to

Re: Test harness options now defined in-tree -- please pull m-c before pushing to try!

2014-03-25 Thread Gregory Szorc
On 3/25/14, 3:14 PM, Andrew Halberstadt wrote: With the completion of bug 981030 test harness options are now defined in-tree. This means any push without this patch [1] will fail. I already went ahead and landed it on all release branches, but if you have an integration or project branch, you m

Re: Test errors

2013-11-19 Thread Ehsan Akhgari
On 2013-11-19 12:16 PM, Neil wrote: "ReferenceError: is is not defined". This is coming from code like: is(a, b, "message"); without the function `is' being defined. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https:/

Re: Test errors

2013-11-19 Thread Ed Morley
On 19 November 2013 17:16:16, Neil wrote: So I was quite surprised to find all manner of junk seems to get reported to the console in a typical test run, including such goodies as "ReferenceError: is is not defined". Why does that not seem to cause a problem yet one particular exception causes a