Hi Ludo -
I think this is very interesting. Thanks for starting the discussion.
I think the only reason it doesn't exist is that no one with time has stepped
forward to claim it.
I've spoken with bsmith in the past about bringing more NSS testing in-house,
but we haven't really had time to think about it. Clearly even just the NSS
part is a major undertaking. If I weren't so busy, I'd consider taking it on.
I do agree, as you mention, that anything UI-based already has an established
home in mozmill.
Best,
Matt Wobensmith
QA Engineer
Mozilla Corporation
Matt Wobensmith
QA Engineer
Mozilla Corporation
----- Original Message -----
From: "Ludovic Hirlimann" <ludo...@mozilla.com>
To: dev-qual...@lists.mozilla.org
Sent: Wednesday, August 3, 2011 10:22:21 AM
Subject: Adding unit and tests to NSS and PSM
Hi,
I've been following NSS and PSM bugs over the last few years. Once in a
while an external contributors come up with a patch - and the review
process is slow mainly because the module owners fear that accepting the
patch might trigger regressions.
This is bad for the module and for the contributor. How can a
contributor stay motivated when a patch takes a very long time to get
into the tree or even to get reviewed.
Back when I started working on Thunderbird we had little to none
automated test. But the developers started adding more and more until we
reached a point where we thought we had enough test that we could
require new checkins to also include test. I've also noticed a decreased
in the time it took for reviews and the higher confidence from reviewers
on patches with tests. Having tests as make Thunderbird less prone to
regression and catching them a matter of hours.
I know, by reading various bug comments, that NSS as some machines
dedicated to testing somewhere. And from what I read It seems that these
tests suites are started manually.
I was wondering why there was no effort into bringing unit tests and
automated testing to the NSS and PSM couple ?
I think adding a automated test infrastructure to NSS and PSM would
make a nice project and would make the life of the current maintainers
a lot easier in the future.
>From what I know this is a not complete list of things that need to happen :
Decide on a unit test frame work for NSS
(<https://secure.wikimedia.org/wikipedia/en/wiki/List_of_unit_testing_frameworks#C>).
Integrate that unit test framework into NSS's build system and repport
failures in either buildbot/tinderbox/ current mozilla build tool (or
other tools used by other consumers of nss).
Start adding test - and from the size and complexity of the code that
seems like a lengthy task.
Implement a policy of adding tests for each new checkins
Make sure the test suite stays green
For PSM it might be bit more simple. As psm is more UI based , we could
reuse the worked done with mozmill and have tests the way Thunderbird as
them. This might mean build a special app that only display's PSM's UI
for testing.
Thoughts ? Did i miss something ?
How complicated do you think such project would be ?
Do you think it would be worth it ?
Ludo
--
Ludovic Hirlimann MozillaMessaging QA lead
https://wiki.mozilla.org/Thunderbird:Testing
http://www.spreadthunderbird.com/aff/79/2
_______________________________________________
dev-quality mailing list
dev-qual...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-quality
--
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto
