On Feb 8, 2008, at 12:03 PM, Charles McCathieNevile wrote:
On Fri, 08 Feb 2008 22:22:59 +0530, Chris Wilson <[EMAIL PROTECTED]
> wrote:
2) In fact, on that note, we're interested to see the test suite be
linked, normatively if necessary.
Yes. I think this is a valuable piece of feedback. Currently W3C
process doesn't require test suites until you're trying to get out
of CR and I think it would be better to have them earlier.
I agree that official test suites should be developed earlier than CR.
Thorough test suites are critical to identifying implementation
issues, and especially important for specs written after the fact for
de facto standard technologies. This is a case where it is hard to
tell if we have met our technical goals without a test suite.
Fortunately, we have unofficial test suites as a starting point.
However, I think that per standard practice the test suite should not
be considered normative, only the text of the spec. In particular,
conformance requirements that are not covered by a test must still be
binding, and in case of conflict between the test suite and the spec,
the spec must win. Of course, if the test suite and the spec ever
disagree we will have to publish bug fixes to the test suite or errata
to the spec, but in the meantime we need to be clear which is normative.
Regards,
Maciej
