On 02/09/13 14:40, Simon Sapin wrote:
Le 02/09/2013 13:04, Sanghyeon Seo a écrit :
Servo can now run some of tests in Official W3C CSS Test Suites.
http://www.w3.org/Style/CSS/Test/
For example, clear-001, 002, 003, 005 pass, and clear-004 fails
because of
paint order (#677).
I would like to integrate passing tests to Servo. What would be the
best way
to do so?
Unfortunately (at least for the CSS 2.1 test suite), very few of these
tests are reftests that can be automated. Most of them require human
confirmation, which is just not viable IMO.
I think it is more valuable to spend time on importing reftests, both
from W3C and Gecko.
In general the best source of cross-browser tests is [1] (for CSS) and
[2] (for the rest of the web platform, excluding JS). These are designed
to work without special magic browser-internal APIs and as such should
work on servo as soon as it supports the relevant features.
The testing work for servo should be focused on getting the automatable
parts of those testsuites running. This includes anything in
web-platform-tests not marked -manual (or at least will once I fix up
the whole repository to match the convention) and almost all of the css3
tests in csswg-test. Actually automating the tests may require some work
on the ability to drive servo externally. For example if one can inject
script from an external process and return data to that process it is
sufficient to run all the javascript-based tests. Alternatively, the
same goal could be met by a way to load urls and a special API for
writing to stdout. If one can navigate to a particular URL and discover
when that load finished (possibly by injecting script, as before), that
is sufficient to run the reftests (one can always use an external
screenshot API rather than cloning the gecko approach of using
extensions to canvas, and indeed this has a number of advantages).
It should also be a goal that any new tests *written* for servo but not
testing the servo internals end up in these repositories rather than in
some servo-specific collection of tests. Fragmentation of testsuites is
a major impediment to interoperability between browsers. The greenfield
implementation of the platform in Servo provides a great opportunity to
find holes in the parts where others are afraid to make changes. Turning
this implementation experience into testcases is a unique opportunity to
find and fix holes in the specs, and create an environment where
everyone is confident that following the spec and passing the related
tests will lead to web-compatible results.
[1] https://github.com/w3c/csswg-test
[2] https://github.com/w3c/web-platform-tests
_______________________________________________
dev-servo mailing list
dev-servo@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-servo
