Re: Standardized assertion methods

2014-06-09 Thread Aryeh Gregor
On Fri, Jun 6, 2014 at 8:35 PM, Boris Zbarsky wrote: > https://tbpl.mozilla.org/?tree=Try&rev=e26ab6d5e1e0 says we have quite a > number of things that are in fact assuming that 5 and "5" should test is(). > I'm not sure how much I like throwing in tons of toString() for that case. Why should

Re: Standardized assertion methods

2014-06-07 Thread L. David Baron
On Wednesday 2014-06-04 14:10 -0400, Boris Zbarsky wrote: > For example, should mochitest-plain be consistent with > mochitest-chrome? I would argue yes; the distinction between which > tests go in which one is more or less arbitrarily decided by what > APIs we do or don't have on SpecialPowers. >

Re: Standardized assertion methods

2014-06-07 Thread L. David Baron
On Friday 2014-06-06 17:42 -0700, Jonas Sicking wrote: > I definitely understand that it'll be a pain to convert existing tests > that rely on the relaxed matching. But rather than making the > implementation of is() be more complex and/or more relaxed, could we > instead convert those tests to eit

Re: Standardized assertion methods

2014-06-07 Thread Gijs Kruitbosch
On 07/06/2014 03:40, Ehsan Akhgari wrote: On 2014-06-06, 4:11 PM, Boris Zbarsky wrote: On 6/6/14, 3:19 PM, Ehsan Akhgari wrote: Can we make is() do those checks explicitly and if neither of these cases apply, fall back to a non-strict equality check? Yes. As in, we could make it special-case

Re: Standardized assertion methods

2014-06-06 Thread Ehsan Akhgari
On 2014-06-06, 10:31 PM, Jonas Sicking wrote: But rather than making the implementation of is() be more complex and/or more relaxed, could we instead convert those tests to either is_relaxed(a, b) or ok(a == b) Yes, we could. It's a largish number of tests (several hundred; a number have mult

Re: Standardized assertion methods

2014-06-06 Thread Ehsan Akhgari
On 2014-06-06, 4:11 PM, Boris Zbarsky wrote: On 6/6/14, 3:19 PM, Ehsan Akhgari wrote: Can we make is() do those checks explicitly and if neither of these cases apply, fall back to a non-strict equality check? Yes. As in, we could make it special-case the number-to-string compare and use == fo

Re: Standardized assertion methods

2014-06-06 Thread Jonas Sicking
On Fri, Jun 6, 2014 at 7:10 PM, Boris Zbarsky wrote: > On 6/6/14, 8:42 PM, Jonas Sicking wrote: >> >> I definitely understand that it'll be a pain to convert existing tests >> that rely on the relaxed matching. > > > Yes. We have about 3300 failing test assertions with is() using ===, and > the v

Re: Standardized assertion methods

2014-06-06 Thread Boris Zbarsky
On 6/6/14, 8:42 PM, Jonas Sicking wrote: I definitely understand that it'll be a pain to convert existing tests that rely on the relaxed matching. Yes. We have about 3300 failing test assertions with is() using ===, and the vast majority are string-vs-number. All I'm saying is that we could

Re: Standardized assertion methods

2014-06-06 Thread Jonas Sicking
On Fri, Jun 6, 2014 at 1:11 PM, Boris Zbarsky wrote: > On 6/6/14, 3:19 PM, Ehsan Akhgari wrote: >> >> Can we make is() do those checks explicitly and if neither of these >> cases apply, fall back to a non-strict equality check? > > > Yes. As in, we could make it special-case the number-to-string

Re: Standardized assertion methods

2014-06-06 Thread Boris Zbarsky
On 6/6/14, 3:19 PM, Ehsan Akhgari wrote: Can we make is() do those checks explicitly and if neither of these cases apply, fall back to a non-strict equality check? Yes. As in, we could make it special-case the number-to-string compare and use == for that, and use === for everything else. -B

Re: Standardized assertion methods

2014-06-06 Thread Ehsan Akhgari
On 2014-06-06, 1:35 PM, Boris Zbarsky wrote: On 6/6/14, 12:21 PM, Boris Zbarsky wrote: That will be the big question, yes. https://tbpl.mozilla.org/?tree=Try&rev=e26ab6d5e1e0 says we have quite a number of things that are in fact assuming that 5 and "5" should test is(). I'm not sure how much

Re: Standardized assertion methods

2014-06-06 Thread Ehsan Akhgari
On 2014-06-05, 2:08 AM, Boris Zbarsky wrote: On 6/4/14, 11:30 PM, Gavin Sharp wrote: - benefits to shared API/implementation seem uncontroversial Agreed. - specifically, consistency between mochitest/SimpleTest-based harnesses (mochitest-plain/mochitest-chrome/mochitest-browser) and xpcshell

Re: Standardized assertion methods

2014-06-06 Thread Boris Zbarsky
On 6/6/14, 12:21 PM, Boris Zbarsky wrote: That will be the big question, yes. https://tbpl.mozilla.org/?tree=Try&rev=e26ab6d5e1e0 says we have quite a number of things that are in fact assuming that 5 and "5" should test is(). I'm not sure how much I like throwing in tons of toString() for

Re: Standardized assertion methods

2014-06-06 Thread Boris Zbarsky
On 6/6/14, 12:21 PM, Boris Zbarsky wrote: Yeah, I've been dealing a lot with edge cases of DOM APIs for the last year or two Case in point. I got some try results, and here's one of the failures: 748 INFO TEST-UNEXPECTED-FAIL | /tests/dom/indexedDB/test/test_multientry.html | exhausted

Re: Standardized assertion methods

2014-06-06 Thread Boris Zbarsky
On 6/6/14, 11:04 AM, Gijs Kruitbosch wrote: I'm interested in the API exposure case (bug link please? :-) I'd have to search, sorry. :( I would have assumed that you would test that kind of thing by checking if the relevant API property is enumerable from the object, rather than checking for

Re: Standardized assertion methods

2014-06-06 Thread Gijs Kruitbosch
On 06/06/2014 15:26, Boris Zbarsky wrote: On 6/6/14, 7:12 AM, Gijs Kruitbosch wrote: No, I think that in 99.99% of cases, I don't care about the type, and therefore I would normally use is() and not care that it's using non-strict equality. I think the case where there is (a) a possibility that

Re: Standardized assertion methods

2014-06-06 Thread Boris Zbarsky
On 6/6/14, 7:12 AM, Gijs Kruitbosch wrote: No, I think that in 99.99% of cases, I don't care about the type, and therefore I would normally use is() and not care that it's using non-strict equality. I think the case where there is (a) a possibility that I could get '5' instead of 5 when code is m

Re: Standardized assertion methods

2014-06-06 Thread Robert O'Callahan
On Fri, Jun 6, 2014 at 11:12 PM, Gijs Kruitbosch wrote: > No, I think that in 99.99% of cases, I don't care about the type, and > therefore I would normally use is() and not care that it's using non-strict > equality. I think the case where there is (a) a possibility that I could > get '5' instea

Re: Standardized assertion methods

2014-06-06 Thread Gijs Kruitbosch
On 06/06/2014 11:56, James Graham wrote: On 06/06/14 11:41, Gijs Kruitbosch wrote: On 06/06/2014 10:29, James Graham wrote: On 05/06/14 10:38, Mike de Boer wrote: As I tried to explain, the CommonJS API naively made sense to me at the time. To others as well, because we’re happily using it. A

Re: Standardized assertion methods

2014-06-06 Thread James Graham
On 06/06/14 11:41, Gijs Kruitbosch wrote: > On 06/06/2014 10:29, James Graham wrote: >> On 05/06/14 10:38, Mike de Boer wrote: >> >>> As I tried to explain, the CommonJS API naively made sense to me at >>> the time. To others as well, because we’re happily using it. As I now >>> understand, some of

Re: Standardized assertion methods

2014-06-06 Thread Gijs Kruitbosch
On 06/06/2014 10:29, James Graham wrote: On 05/06/14 10:38, Mike de Boer wrote: As I tried to explain, the CommonJS API naively made sense to me at the time. To others as well, because we’re happily using it. As I now understand, some of us are very attached to a specific, different, API. FWI

Re: Standardized assertion methods

2014-06-06 Thread Jonas Sicking
On Fri, Jun 6, 2014 at 2:29 AM, James Graham wrote: > * All the todo stuff is mixing concerns. It forces you into a mode of > test writing where properties of a single implementation are hardcoded > into the testcases. This isn't a huge problem when there is only a > single relevant implementation

Re: Standardized assertion methods

2014-06-06 Thread James Graham
On 05/06/14 10:38, Mike de Boer wrote: > As I tried to explain, the CommonJS API naively made sense to me at > the time. To others as well, because we’re happily using it. As I now > understand, some of us are very attached to a specific, different, > API. FWIW I don't think that I am attached to

Re: Standardized assertion methods

2014-06-05 Thread Gavin Sharp
On Thu, Jun 5, 2014 at 3:33 AM, Mike de Boer wrote: > I want that to happen, which is most ‘cost-efficient’; if that’s moving the > implementation of `is`, `isnot`, `ise` to a separate module whilst keeping > the method names available to all Mochitest(-browser) tests, than we do that. > > If pr

Re: Standardized assertion methods

2014-06-05 Thread Mike de Boer
On 05 Jun 2014, at 12:00, Dao wrote: > On 05.06.2014 11:38, Mike de Boer wrote: >> On 05 Jun 2014, at 09:54, Dao wrote: >> >>> On 04.06.2014 11:45, Mike de Boer wrote: The reason CommonJS came into view was not because of it’s semantic superiority, but because of its similarity to bo

Re: Standardized assertion methods

2014-06-05 Thread Dao
On 05.06.2014 11:38, Mike de Boer wrote: On 05 Jun 2014, at 09:54, Dao wrote: On 04.06.2014 11:45, Mike de Boer wrote: The reason CommonJS came into view was not because of it’s semantic superiority, but because of its similarity to both the XPCShell-test and Mochitest assertion styles and

Re: Standardized assertion methods

2014-06-05 Thread Mike de Boer
On 05 Jun 2014, at 09:54, Dao wrote: > On 04.06.2014 11:45, Mike de Boer wrote: >> The reason CommonJS came into view was not because of it’s semantic >> superiority, but because of its similarity to both the XPCShell-test and >> Mochitest assertion styles and implementation. >> This way I thou

Re: Standardized assertion methods

2014-06-05 Thread Dao
On 05.06.2014 09:54, Dao wrote: On 04.06.2014 11:45, Mike de Boer wrote: The reason CommonJS came into view was not because of it’s semantic superiority, but because of its similarity to both the XPCShell-test and Mochitest assertion styles and implementation. This way I thought we could circumv

Re: Standardized assertion methods

2014-06-05 Thread Dao
On 04.06.2014 11:45, Mike de Boer wrote: The reason CommonJS came into view was not because of it’s semantic superiority, but because of its similarity to both the XPCShell-test and Mochitest assertion styles and implementation. This way I thought we could circumvent ppl to get worried about re

Re: Standardized assertion methods

2014-06-04 Thread Boris Zbarsky
On 6/4/14, 11:30 PM, Gavin Sharp wrote: - benefits to shared API/implementation seem uncontroversial Agreed. - specifically, consistency between mochitest/SimpleTest-based harnesses (mochitest-plain/mochitest-chrome/mochitest-browser) and xpcshell tests is what we care about primarily. I don'

Re: Standardized assertion methods

2014-06-04 Thread Gavin Sharp
I still don't believe either of you :) Obviously my position isn't "let's make it it more frustrating to write tests"; I think you're both vastly overstating the costs of switching to a slightly different, similar API. Any change is initially jarring, but I just don't buy that this change would cau

Re: Standardized assertion methods

2014-06-04 Thread Boris Zbarsky
On 6/4/14, 2:21 PM, James Graham wrote: * They encourage the use of deepEqual which has underdefined semantics, particularly in the case of objects that contain cycles (it looks like Assert.jsm goes into an infinite loop in this case, but I may have misread the code). You didn't misread the cod

Re: Standardized assertion methods

2014-06-04 Thread James Graham
On 04/06/14 18:42, Mike de Boer wrote: > On 04 Jun 2014, at 19:20, Ehsan Akhgari > wrote: > >> On 2014-06-04, 5:45 AM, Mike de Boer wrote: >>> On 04 Jun 2014, at 00:33, James Graham >>> wrote: >>> On 03/06/14 20:34, Boris Zbarsky wrote: > I'm arguing against Assert.jsm using the

Re: Standardized assertion methods

2014-06-04 Thread Boris Zbarsky
On 6/4/14, 1:56 PM, Ted Mielczarek wrote: The inconsistency between our various test harnesses makes it harder than necessary to write different types of tests. Yes, agreed. RE: the discussion of testharness.js etc, I think those are even farther afield, since the testing model there is much

Re: Standardized assertion methods

2014-06-04 Thread Boris Zbarsky
On 6/4/14, 1:42 PM, Mike de Boer wrote: I wasn’t implying that they’re broken at all, it’s just that James was hinting at that. Our existing testing frameworks are broken in terms of the goals of the testharness framework, as far as I understand. For example, one of the primary goals of tes

Re: Standardized assertion methods

2014-06-04 Thread Ehsan Akhgari
On 2014-06-04, 1:42 PM, Mike de Boer wrote: On 04 Jun 2014, at 19:20, Ehsan Akhgari mailto:ehsan.akhg...@gmail.com>> wrote: On 2014-06-04, 5:45 AM, Mike de Boer wrote: On 04 Jun 2014, at 00:33, James Graham mailto:ja...@hoppipolla.co.uk>> wrote: On 03/06/14 20:34, Boris Zbarsky wrote: I'm

Re: Standardized assertion methods

2014-06-04 Thread Ted Mielczarek
On 6/4/2014 1:20 PM, Ehsan Akhgari wrote: > The other thing that you need to note is that there is years of > experience behind each one of our test frameworks, and there are > probably several hundred thousand lines of code written against any of > them. And there are many many people who have be

Re: Standardized assertion methods

2014-06-04 Thread Mike de Boer
On 04 Jun 2014, at 19:20, Ehsan Akhgari wrote: > On 2014-06-04, 5:45 AM, Mike de Boer wrote: >> On 04 Jun 2014, at 00:33, James Graham wrote: >> >>> On 03/06/14 20:34, Boris Zbarsky wrote: >>> I'm arguing against Assert.jsm using the commonjs API names. >>> >>> And I am arguing against u

Re: Standardized assertion methods

2014-06-04 Thread Ehsan Akhgari
On 2014-06-04, 5:45 AM, Mike de Boer wrote: On 04 Jun 2014, at 00:33, James Graham wrote: On 03/06/14 20:34, Boris Zbarsky wrote: I'm arguing against Assert.jsm using the commonjs API names. And I am arguing against using the CommonJS semantics. If we are adding new assertions it shouldn't

Re: Standardized assertion methods

2014-06-04 Thread Mike de Boer
On 04 Jun 2014, at 00:33, James Graham wrote: > On 03/06/14 20:34, Boris Zbarsky wrote: > >> I'm arguing against Assert.jsm using the commonjs API names. > > And I am arguing against using the CommonJS semantics. If we are adding > new assertions it shouldn't be ones that encourage broken tests

Re: Standardized assertion methods

2014-06-03 Thread James Graham
On 03/06/14 20:34, Boris Zbarsky wrote: > I'm arguing against Assert.jsm using the commonjs API names. And I am arguing against using the CommonJS semantics. If we are adding new assertions it shouldn't be ones that encourage broken tests. ___ dev-plat

Re: Standardized assertion methods

2014-06-03 Thread James Graham
On 03/06/14 22:28, Jonas Sicking wrote: > testharness.js still requires lots of boiler plate. Especially when > writing async tests. And especially if you try to follow the rule that > each test within a file should clean up after itself. At this point testharness.js has taken several steps to al

Re: Standardized assertion methods

2014-06-03 Thread Jonas Sicking
On Tue, Jun 3, 2014 at 12:41 PM, Bobby Holley wrote: > On Tue, Jun 3, 2014 at 12:31 PM, Boris Zbarsky wrote: > >> I can certainly buy "it's longer than what I'm >>> used to", and even "incremental effort is required" - just not >>> "incremental effort is required and that effort is non-negligible

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 3:57 PM, Ehsan Akhgari wrote: Nobody is opposing a proposal where someone volunteered to create the rewriting facilities you're mentioning and using them to fix the entire code base in one go. I somewhat am. -Boris ___ dev-platform mailin

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 3:31 PM, Boris Zbarsky wrote: Maybe that's a personal failing of mine, but I suspect not. More precisely, I expect it's a personal failing that is widespread, not just my personal little quirk. -Boris ___ dev-platform mailing list dev-pl

Re: Standardized assertion methods

2014-06-03 Thread Ehsan Akhgari
On 2014-06-03, 3:34 PM, Boris Zbarsky wrote: On 6/3/14, 2:36 PM, Gregory Szorc wrote: There is a clear win in the ability to reuse, understand, and modify the common code. No one is arguing against having common harness code as far as I can see. I can't even recall which file(s) contain is/o

Re: Standardized assertion methods

2014-06-03 Thread Bobby Holley
On Tue, Jun 3, 2014 at 12:31 PM, Boris Zbarsky wrote: > I can certainly buy "it's longer than what I'm >> used to", and even "incremental effort is required" - just not >> "incremental effort is required and that effort is non-negligible >> given other factors" >> > > Purely subjectively, it's no

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 2:36 PM, Gregory Szorc wrote: There is a clear win in the ability to reuse, understand, and modify the common code. No one is arguing against having common harness code as far as I can see. I can't even recall which file(s) contain is/ok from mochitest SimpleTest.js. Which also

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 1:49 PM, Gavin Sharp wrote: I suspected this is where we'd end up :) "Reasonability" is just as subjective as aesthetics. Of cours. I really have a hard time accepting at face value the argument "Assert.notEqual (or other shorter variants) is unreasonably long to type/paste repeate

Re: Standardized assertion methods

2014-06-03 Thread Gregory Szorc
On 6/3/14, 11:28 AM, Ehsan Akhgari wrote: On 2014-06-03, 2:17 PM, Gavin Sharp wrote: I won't argue that a great case has been made :) But I see inherent value in consistency (both in the implementations and in the user-exposed API) for assertions across our in-tree test suites (or at least, acro

Re: Standardized assertion methods

2014-06-03 Thread Ehsan Akhgari
On 2014-06-03, 2:17 PM, Gavin Sharp wrote: I won't argue that a great case has been made :) But I see inherent value in consistency (both in the implementations and in the user-exposed API) for assertions across our in-tree test suites (or at least, across mochitest-based harnesses and xpcshell).

Re: Standardized assertion methods

2014-06-03 Thread Gavin Sharp
I won't argue that a great case has been made :) But I see inherent value in consistency (both in the implementations and in the user-exposed API) for assertions across our in-tree test suites (or at least, across mochitest-based harnesses and xpcshell). Do you disagree? Gavin On Tue, Jun 3, 201

Re: Standardized assertion methods

2014-06-03 Thread Ehsan Akhgari
On 2014-06-03, 1:49 PM, Gavin Sharp wrote: On Mon, Jun 2, 2014 at 9:35 PM, Boris Zbarsky wrote: I think what xpcshell has now and what testharness says and what's being proposed (with the "Assert." prefix) are unreasonably long/verbose. I suspected this is where we'd end up :) "Reasonability"

Re: Standardized assertion methods

2014-06-03 Thread Gavin Sharp
On Mon, Jun 2, 2014 at 9:35 PM, Boris Zbarsky wrote: > I think what xpcshell has now and what testharness says and what's being > proposed (with the "Assert." prefix) are unreasonably long/verbose. I suspected this is where we'd end up :) "Reasonability" is just as subjective as aesthetics. I re

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 11:17 AM, Joshua Cranmer 🐧 wrote: I'm used to xpcshell tests more than mochitests, and the biggest difference by far between xpcshell and mocha that I'm aware of is that mocha counts tests at finer granularity: xpcshell tests work on a file-by-file basis, whereas mocha tests work at th

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 11:19 AM, L. David Baron wrote: Locally I generally want to see all the failures, since seeing the complete set of failures is often a much better hint as to the cause of the failures than just seeing the first one. Yes, exactly. -Boris _

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 9:29 AM, Mike de Boer wrote: Nope, you got me there - I generalised too easily. This statement is based on personal experience, not science. I'm not looking for science, necessarily. I'm looking for an understanding of the problems we're trying to solve. My basic issue is that f

Re: Standardized assertion methods

2014-06-03 Thread Ehsan Akhgari
On 2014-06-03, 12:13 PM, Mike de Boer wrote: On 03 Jun 2014, at 17:39, Ehsan Akhgari wrote: On 2014-06-02, 9:35 PM, Boris Zbarsky wrote: On 6/2/14, 5:33 PM, Gavin Sharp wrote: Do either of you have reasoning for that other than "it looks better to me"? My personal experience is that when I

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
On 03 Jun 2014, at 17:39, Ehsan Akhgari wrote: > On 2014-06-02, 9:35 PM, Boris Zbarsky wrote: >> On 6/2/14, 5:33 PM, Gavin Sharp wrote: >>> Do either of you have reasoning for that other than "it looks better >>> to me"? >> >> My personal experience is that when I try to write xpcshell tests the

Re: Standardized assertion methods

2014-06-03 Thread Gregory Szorc
On 6/3/14, 6:16 AM, Mike de Boer wrote: Writing wrappers in python around things to improve the current situation like a band-aid isn’t the way I’m used to fix things; I like to take the bull by the horns[1] I’d like to ask _why_ structured logging needs to be bolted on top of what we have cu

Re: Standardized assertion methods

2014-06-03 Thread Ehsan Akhgari
On 2014-06-02, 9:35 PM, Boris Zbarsky wrote: On 6/2/14, 5:33 PM, Gavin Sharp wrote: Do either of you have reasoning for that other than "it looks better to me"? My personal experience is that when I try to write xpcshell tests the amount of time it takes to type the test function names is very

Re: Standardized assertion methods

2014-06-03 Thread L. David Baron
On Tuesday 2014-06-03 15:21 +0200, Mike de Boer wrote: > On 03 Jun 2014, at 15:07, Boris Zbarsky wrote: > > I assume that the mochitest version will use a different reporter that > > doesn't throw-and-terminate, to preserve the current semantics of mochitest > > assertions. (If this assumption

Re: Standardized assertion methods

2014-06-03 Thread Joshua Cranmer 🐧
On 6/3/2014 8:39 AM, Gijs Kruitbosch wrote: On 03/06/2014 14:16, Mike de Boer wrote: Indeed, I’m used to the NodeJS/ Mocha flow of writing tests as fast, or even faster, as writing the implementation of a feature. I could group tests, isolate one, hook in a debugger at any point and more. This

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
James, thanks so much for the additional background information about testing at Mozilla. I’m currently following the bugs you mentioned earlier and am looking forward to their results! Mike. On 03 Jun 2014, at 16:07, James Graham wrote: > On 03/06/14 14:16, Mike de Boer wrote: > >> Writing

Future improvements to Javascript unit-test runners (was: Standardized assertion methods)

2014-06-03 Thread Mike de Boer
I started to summarise the things I’d like to see in a JS unit test runner here[1]: * mini-core. * Async support as a base. We’ve added `add_task()` methods where possible, but we haven’t made it a core feature of the legacy suites in use today. Generators yielding Promises are now possible,

Re: Standardized assertion methods

2014-06-03 Thread James Graham
On 03/06/14 14:16, Mike de Boer wrote: > Writing wrappers in python around things to improve the current > situation like a band-aid isn’t the way I’m used to fix things; I > like to take the bull by the horns[1] > > I’d like to ask _why_ structured logging needs to be bolted on top of > what we

Re: Standardized assertion methods

2014-06-03 Thread Gijs Kruitbosch
I don't think this is off-topic. I'm essentially asking: why did you focus on this, and why in this way? More broadly, I'm asking what it is you're missing from node/mocha. If you think that needs its own topic, feel free to fork the summary. In any case, discussions about the ease of use of o

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
I understand all that and I *think* you missed the header mentioning I was going off-topic… Mike. On 03 Jun 2014, at 15:39, Gijs Kruitbosch wrote: > On 03/06/2014 14:16, Mike de Boer wrote: >> Indeed, I’m used to the NodeJS/ Mocha flow of writing tests as fast, or even >> faster, as writing t

Re: Standardized assertion methods

2014-06-03 Thread Gijs Kruitbosch
On 03/06/2014 14:16, Mike de Boer wrote: Indeed, I’m used to the NodeJS/ Mocha flow of writing tests as fast, or even faster, as writing the implementation of a feature. I could group tests, isolate one, hook in a debugger at any point and more. This is something I miss while working on Fx and

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
On 03 Jun 2014, at 14:50, Boris Zbarsky wrote: > On 6/3/14, 6:22 AM, Mike de Boer wrote: >> Their lack of modularity costs us flexibility in adopting and/ or promoting >> TDD development. > > Mike, I'm very curious about this part. Do you have a link offhand to a more > detailed explanation o

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
On 03 Jun 2014, at 15:07, Boris Zbarsky wrote: > On 6/3/14, 8:50 AM, Boris Zbarsky wrote: >> I do think we should be very intentional about adopting something new, >> both in terms of semantics (mochitest is() using == is a mistake we >> should not duplicate in the short-name comparison function

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
On 03 Jun 2014, at 14:54, Boris Zbarsky wrote: > On 6/3/14, 8:39 AM, Mike de Boer wrote: >> I think it helps provide a common, immediate understanding for new >> contributors who’d like to write test for the code they contribute, as the >> add-on SDK and the NodeJS community already use it excl

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 8:50 AM, Boris Zbarsky wrote: I do think we should be very intentional about adopting something new, both in terms of semantics (mochitest is() using == is a mistake we should not duplicate in the short-name comparison function in the new setup) y One other note. The checkin so far

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 6:22 AM, Mike de Boer wrote: Their lack of modularity costs us flexibility in adopting and/ or promoting TDD development. Mike, I'm very curious about this part. Do you have a link offhand to a more detailed explanation of the issues here? Note that none of us think Mochitest is

Re: Standardized assertion methods

2014-06-03 Thread Boris Zbarsky
On 6/3/14, 8:39 AM, Mike de Boer wrote: I think it helps provide a common, immediate understanding for new contributors who’d like to write test for the code they contribute, as the add-on SDK and the NodeJS community already use it exclusively. I think there's a bit of functional area bias h

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
Indeed I meant to say the following: Taking the CommonJS spec as an umbrella for these simple assertion methods is more of a nice side-effect than it was the primary objective we started off with. I think it helps provide a common, immediate understanding for new contributors who’d like to writ

Re: Standardized assertion methods

2014-06-03 Thread James Graham
On 03/06/14 12:27, Mike de Boer wrote: >>> 4. None of the test-suites promote modularity and needlessly dictate >>> a reporting style. What I mean by this is that there’s no way to hook >>> different reporting styles in a test runner to promote TDD, for >>> example. What does automation use to det

Re: Standardized assertion methods

2014-06-03 Thread Gijs Kruitbosch
On 03/06/2014 12:27, Mike de Boer wrote: 5. Assertion semantics are indeed poorly specified, across the board. Our switch from `do_check_matches()` to `deepEqual()` even revealed a buggy implementation there, which we didn’t know about. Apart from that, it was largely undocumented, not fully co

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
Please see my comments inline. On 03 Jun 2014, at 12:57, James Graham wrote: > I'm not sure I grasp your overall point, but I have a few comments. > > On 03/06/14 11:22, Mike de Boer wrote: >> 1. The `Assert.*` namespace is optional and may be omitted. This >> module is also present in the addo

Re: Standardized assertion methods

2014-06-03 Thread James Graham
I'm not sure I grasp your overall point, but I have a few comments. On 03/06/14 11:22, Mike de Boer wrote: > 1. The `Assert.*` namespace is optional and may be omitted. This > module is also present in the addon-sdk and used _with_ that > namespace, usually with a lowercase `assert.*`. Please pick

Re: Standardized assertion methods

2014-06-03 Thread Mike de Boer
1. The `Assert.*` namespace is optional and may be omitted. This module is also present in the addon-sdk and used _with_ that namespace, usually with a lowercase `assert.*`. Please pick whatever suits your fancy. 2. testharness.js, Mochitest, XPCShell’s head.js and other suite-runners that we u

Re: Standardized assertion methods

2014-06-03 Thread James Graham
On 03/06/14 00:24, Chris Peterson wrote: > On 6/2/14, 3:42 PM, Ehsan Akhgari wrote: >> 2. I also value consistency more than my personal preferences, and based >> on that, using the existing APIs in some tests and the new APIs in other >> tests (even if we agreed that #1 above doesn't matter) is st

Re: Standardized assertion methods

2014-06-02 Thread Dao
On 03.06.2014 00:42, Ehsan Akhgari wrote: Also, I'm not sure where the original discussion happened, Ditto. For a change that affects pretty much all mozilla-central hackers, I would have expected a public proposal and a feedback round on this list. (My initial reaction was similar to Ehsan

Re: Standardized assertion methods

2014-06-02 Thread Boris Zbarsky
On 6/2/14, 9:44 PM, Boris Zbarsky wrote: On 6/2/14, 7:26 PM, Martin Thomson wrote: Sounds like an area ripe for standardisation Note that standardizing several test suites on the same API might not work all that well when they have different goals and operating parameters Specifically, testh

Re: Standardized assertion methods

2014-06-02 Thread Boris Zbarsky
On 6/2/14, 7:26 PM, Martin Thomson wrote: Sounds like an area ripe for standardisation Note that standardizing several test suites on the same API might not work all that well when they have different goals and operating parameters, because what can you end up with is an API that doesn't qui

Re: Standardized assertion methods

2014-06-02 Thread Boris Zbarsky
On 6/2/14, 5:33 PM, Gavin Sharp wrote: Do either of you have reasoning for that other than "it looks better to me"? My personal experience is that when I try to write xpcshell tests the amount of time it takes to type the test function names is very noticeable and actively interrupts my think

Re: Standardized assertion methods

2014-06-02 Thread Martin Thomson
On 2014-06-02, at 16:24, Chris Peterson wrote: > btw, in the mozilla.dev.tech.javascript-engine.internals fork of this thread, > bz and David Bruant pointed out that W3C's testharness and TC39's test262 > each use yet another set of assertion function names. Any tests we import > from those te

Re: Standardized assertion methods

2014-06-02 Thread Chris Peterson
On 6/2/14, 3:42 PM, Ehsan Akhgari wrote: 2. I also value consistency more than my personal preferences, and based on that, using the existing APIs in some tests and the new APIs in other tests (even if we agreed that #1 above doesn't matter) is strictly worse than the status quo. btw, in the mo

Re: Standardized assertion methods

2014-06-02 Thread Ehsan Akhgari
On 2014-06-02, 5:33 PM, Gavin Sharp wrote: Do either of you have reasoning for that other than "it looks better to me"? I personally think consistency trumps any personal preferences based on length/concision, as long as what we end up with isn't unreasonably long/verbose. I have two reasons:

Re: Standardized assertion methods

2014-06-02 Thread Gavin Sharp
Do either of you have reasoning for that other than "it looks better to me"? I personally think consistency trumps any personal preferences based on length/concision, as long as what we end up with isn't unreasonably long/verbose. Gavin On Mon, Jun 2, 2014 at 4:47 PM, Ehsan Akhgari wrote: > On M

Re: Standardized assertion methods

2014-06-02 Thread Ehsan Akhgari
On Mon, Jun 2, 2014 at 4:14 PM, Boris Zbarsky wrote: > On 6/2/14, 3:34 PM, Paolo Amadini wrote: > >> It seems to me that if we don't have external compatibility needs, we >> might as well move mochitests to use a set of assertion methods that is >> the same as xpcshell and maybe other test suites

Re: Standardized assertion methods

2014-06-02 Thread Boris Zbarsky
On 6/2/14, 3:34 PM, Paolo Amadini wrote: It seems to me that if we don't have external compatibility needs, we might as well move mochitests to use a set of assertion methods that is the same as xpcshell and maybe other test suites. Yes, but imho we should be moving xpcshell in the direction of

Re: Standardized assertion methods

2014-06-02 Thread Paolo Amadini
On 6/2/2014 7:08 PM, Ehsan Akhgari wrote: >> On 6/2/2014 4:59 PM, Ehsan Akhgari wrote: >>> I'm _pretty_ sure that the answer is no for >>> mochitest-chrome at least. >> >> Are we running these tests out-of-tree in other environments? > > Do you mean by just opening the page in the browser? I don'

Re: Standardized assertion methods

2014-06-02 Thread Ehsan Akhgari
On Mon, Jun 2, 2014 at 12:27 PM, Paolo Amadini wrote: > On 6/2/2014 4:59 PM, Ehsan Akhgari wrote: > > I'm _pretty_ sure that the answer is no for > > mochitest-chrome at least. > > Are we running these tests out-of-tree in other environments? Do you mean by just opening the page in the browser?

Re: Standardized assertion methods

2014-06-02 Thread Paolo Amadini
On 6/2/2014 4:51 PM, Gijs Kruitbosch wrote: > Concretely, IMO in the code you cite there should be a blank line before > each of the 'parent' reassignments. I definitely agree, and I would also use the "Assert." prefix to make the separation between action and check clearer (while if I understand

Re: Standardized assertion methods

2014-06-02 Thread Paolo Amadini
On 6/2/2014 4:59 PM, Ehsan Akhgari wrote: > I'm _pretty_ sure that the answer is no for > mochitest-chrome at least. Are we running these tests out-of-tree in other environments? Paolo ___ dev-platform mailing list dev-platform@lists.mozilla.org https:/

Re: Standardized assertion methods

2014-06-02 Thread Mike de Boer
On 02 Jun 2014, at 17:39, Paolo Amadini wrote: > > Have you considered requiring test cases to use the the "Assert." > namespace explicitly? I would find that style more readable, and also > assertions easier to find when scanning the code. And they're still > shorter than current assertion funct

Re: Standardized assertion methods

2014-06-02 Thread Ehsan Akhgari
mochitest-browser and mochitest-chrome have traditionally used the SimpleTest APIs. I'm not aware of the benefits that Assert.jsm provides, but do we really want to diverge these APIs with mochitest-plain? I'm _pretty_ sure that the answer is no for mochitest-chrome at least. Can you please

Re: Standardized assertion methods

2014-06-02 Thread Gijs Kruitbosch
Warning: pet peeve coming up. I agree we should separate these things more clearly. I don't think making people type the same 7 characters repeatedly is a good way to do that. I think we should be more liberal with using blank lines instead. It's too often I see 30-line blocks of code with n

Re: Standardized assertion methods

2014-06-02 Thread Paolo Amadini
On 6/2/2014 11:37 AM, Mike de Boer wrote: > Since last Friday[3], each assertion method in Assert.jsm is available in the > global scope of a unit test as well. > Now we can say that the ‘old’ XPCShell-test assertion methods are deprecated > in favour of the Assert.jsm ones. I think it's a very

  1   2   >