On Mon, Feb 25, 2013 at 12:52 PM, Dirk Pranke <[email protected]> wrote:
> On Mon, Feb 25, 2013 at 12:38 PM, Eric Seidel <[email protected]> wrote: > > I've noticed as of late several different approaches being used when > > adding/changing LayoutTests which need rebaselining on other platforms. > > > > Obviously we cannot expect developers to test/rebaseline on all platforms > > before landing given our current infrastructure. > > > > But what should we expect them to do? > > > > Currently some folks add failing expectations to other ports > > TestExpectations. Some add [skip]. Some even add them to the (new) > global > > TestExpectations file. > > > > What's the proper course of action? > > > > I checked: > > http://trac.webkit.org/wiki/Keeping%20the%20Tree%20Green > > > http://trac.webkit.org/wiki/Creating%20and%20Submitting%20Layout%20Tests%20and%20Patches > > http://trac.webkit.org/wiki/TestExpectations > > and didn't see the "I expect this to fail/need rebaseline on other ports" > > case discussed. > > > > Memory says that the last time this was raised [1], the consensus was > to land the change, turn the tree red, notify as many gardeners / port > maintainers as possible, and rebaseline as quickly as possible. I.e., > don't add entries to TestExpectations. Of course, such a policy > wouldn't play nicely w/ the EWS bots, and I think this discussion > predated everyone switching over to TestExpectations (and certainly > predated the generic expectations file). > > I don't find the above entirely satisfactory, but I also don't have > any great alternatives to endorse given the existing tooling. > > Also, if the test fails generically (all ports), it probably shouldn't > be landed. I'm not sure why you couldn't at least update the port > you're developing on; if you don't, how do you know your fix is > working at all? (Of course, there could be a case where a > high-priority fix might break some low-priority tests). > Adding a new feature behind a build flag disabled everywhere by default. > I remember some discussion of a [rebaseline] keyword in TestExpectations, > > but I'm not sure that ever made it in? > > The [ NeedsRebaseline ] enhancement is, as of yet, unimplemented and > unclaimed [2]. It shouldn't be too hard for someone to try it if > they're looking for a reason to explore the NRWT code :) > I object to adding such a thing. People add and forget about these entries way too often: # Needs rebaseline fast/sub-pixel/inline-block-with-padding.html [ Failure ] # Rebaseline required after https://bugs.webkit.org/show_bug.cgi?id=101177 webkit.org/b/101177svg/dynamic-updates/SVGUseElement-dom-requiredFeatures.html [ ImageOnlyFailure Pass ] webkit.org/b/101177 svg/repaint/inner-svg-change-viewBox.svg [ ImageOnlyFailure Pass ] # These tests need to be rebaselined after https://bugs.webkit.org/show_bug.cgi?id=99984 webkit.org/b/99984 svg/as-image/image-preserveAspectRatio-all.svg [ ImageOnlyFailure Pass ] webkit.org/b/99984 svg/filters/feImage-preserveAspectRatio-all.svg [ ImageOnlyFailure Pass ] webkit.org/b/99984svg/dynamic-updates/SVGFEImageElement-dom-preserveAspectRatio-attr.html [ ImageOnlyFailure Pass ] webkit.org/b/99984svg/dynamic-updates/SVGFEImageElement-svgdom-preserveAspectRatio-prop.html [ ImageOnlyFailure Pass ] # Needs rebaseline after bug https://bugs.webkit.org/show_bug.cgi?id=104462 webkit.org/b/98687fast/css-generated-content/table-row-group-to-inline.html [ Failure ] webkit.org/b/104595 fast/css/empty-generated-content.html [ Failure ] # Needs a rebaseline after WK108429 lands. webkit.org/b/108429 svg/custom/text-ctm.svg [ Pass Failure ] # Transient. Needs rebaseline. webkit.org/b/103955 fast/repaint/caret-with-transformation.html [ Missing ] # Needs rebaseline. webkit.org/b/105574 editing/input/caret-at-the-edge-of-contenteditable.html [ Failure ] webkit.org/b/105574editing/input/reveal-caret-of-multiline-contenteditable.html [ Failure ] webkit.org/b/105574 editing/input/reveal-caret-of-multiline-input.html [ Failure ] webkit.org/b/105574fast/spatial-navigation/snav-div-overflow-scrol-hidden.html [ Failure ] # Rebaseline required after https://webkit.org/b/31397 webkit.org/b/107567 svg/batik/text/xmlSpace.svg [ Failure ] webkit.org/b/107567 fast/text/capitalize-empty-generated-string.html [ Failure ] webkit.org/b/107567 fast/text/whitespace/007.html [ Failure ] webkit.org/b/107567 fast/text/whitespace/006.html [ Failure ] webkit.org/b/107567 fast/inline/drawStyledEmptyInlinesWithWS.html [ Failure ] webkit.org/b/107567 tables/mozilla/bugs/bug1318.html [ Failure ] webkit.org/b/107567 fast/inline/drawStyledEmptyInlines.html [ Failure ] webkit.org/b/107567 tables/mozilla/bugs/bug113235-3.html [ Failure ] webkit.org/b/107567 css2.1/t0505-c16-descendant-01-e.html [ Failure ] webkit.org/b/107567 tables/mozilla/bugs/bug1188.html [ Failure ] webkit.org/b/107567 editing/selection/extend-by-sentence-001.html [ Failure ] webkit.org/b/107567 svg/carto.net/combobox.svg [ Failure ] # Needs rebaseline after https://bugs.webkit.org/show_bug.cgi?id=70634 webkit.org/b/109507compositing/contents-opaque/contents-opaque-background-color.html [ Failure ] webkit.org/b/109507compositing/contents-opaque/contents-opaque-layer-opacity.html [ Failure ] webkit.org/b/109507compositing/contents-opaque/contents-opaque-layer-transform.html [ Failure ] webkit.org/b/109507 compositing/rtl/rtl-fixed.html [ Failure ] webkit.org/b/109507 compositing/rtl/rtl-fixed-overflow.html [ Failure ] # Rebaseline required after https://webkit.org/b/95772 webkit.org/b/109209fast/text/international/bidi-ignored-for-first-child-inline.html [ Failure ] - R. Niwa
_______________________________________________ webkit-dev mailing list [email protected] https://lists.webkit.org/mailman/listinfo/webkit-dev
