On Mon, Feb 25, 2013 at 6:28 PM, Silvia Pfeiffer <[email protected]>wrote:
> On Tue, Feb 26, 2013 at 9:34 AM, Ryosuke Niwa <[email protected]> wrote: > >> It should be fairly straight forward to create a tool that analyzes files >> changed in each commit and deduce which tests' expected results have been >> changed. The tool can then fetch results from each port' bot for those >> tests and automatically land them. It can then comment on the bug >> automatically about these rebaseline commits. There is no need to add & >> remove entries from TestExpectation files. >> > > The consequence of this is that it's possible that rebaselines get > committed that may actually indicate a new bug. Since your suggestion is to > attach this to the bug through which the patch landed, I think that's ok > and the original author (or the reviewer) should feel obliged to follow up > on the newly introduced bug by creating a new bug. > > If we can do this in a way that avoids the bots from going red, I think > it's a good solution. > I don't like a dogmatic approach like keeping bots green at all cost. We need to weigh costs of bots turning red versus asking every contributor to do extra things in order to keep bots green while noticing that some of our 400 contributors are bound to make mistakes every now and then. After all, those people who get affected by red bots are also people who submit patches and (hopefully) maintain bots. I'm also saddened by the fact that the first thing some people do when they see builds failing or some tests failing is to blame others or complain instead of trying to fix those problems themselves. - R. Niwa
_______________________________________________ webkit-dev mailing list [email protected] https://lists.webkit.org/mailman/listinfo/webkit-dev
