Diego Elio Pettenò posted on Mon, 05 Nov 2012 07:39:19 -0800 as excerpted:
> On 05/11/2012 07:31, Steven J. Long wrote: >> Are you really missing the fact that by testing someone's overlay, the >> package would by definition not be in the tree, and you wouldn't have >> to file any bugs at all, just (automatically) email the output back to >> the overlay developer? > > Which means I wouldn't be filing bugs for the problems with the > _existing_ packages that are in tree, which is what the users actually > _use_ by default. > > If the users are forced to use overlays to get working packages, then I > feel I'm perfectly right with screaming at the developers who are using > overlays for development because they leave users in a sorry state. I'm not sure if this is what SL had in mind, but it's what I thought of when I read his suggestion, triggered here by the "won't have to file bugs at all" bit. What about doing overlays, but ONLY one-at-a-time, and ONLY on special- request-runs, presumably immediately pre-tree-introduction? Among other things that might help for stuff like kde where a whole slew of packages are introduced to the tree (and should be tested) together, something that's easier done from a staging overlay. That way, any overlay runs would be pre-requested by the overlay person/ project, so they'd be specifically expecting results. Thus the "could (if desired) skip filing bugs, just direct-mail" part SL mentioned, or automated bug filing but in this case it'd be ENTIRELY by request and expected, since they would have pre-arranged for the tinderbox run and would be expecting/prepared-for the results. Additionally, one-overlay-at-a-time would automatically mean what's ever there is tested only against the tree and other packages in that overlay, limiting the "where'd it come from" problem. In fact, a pre-arranged unmask file (say tinderbox.unmask) in the profile could be used effectively as a whitelist to specify specific packages and versions for the tinderbox run. If it's in the whitelist, it's from the overlay, otherwise it's from the tree. This would let the overlay folks keep their live-builds, etc, out of tinderbox run, since they're presumably not tinderbox-test-ready anyway, as well as nicely solving the "where'd it come from" question. And tree packages would still be heavily emphasized, since all testing except for other packages in that specific overlay test would be against the tree. > Let me try to re-explain this to you: let's say I import overlay $foobar > and $foobar has library A and packages B C D E and F. Now there's > package G in main tree that also uses library A. It fails in the > tinderbox run. Who I have to report it to? This one would then be easy. Package G is in the main tree but you're doing a specifically requested overlay run, and it's failing built against overlay library A. You file the bug with the overlay maintainer, since that's specifically what you're testing, and library A is in tinderbox.unmask so the overlay version is specifically being tested in this run. But here's the kicker. Now you're more or less simply providing the tinderbox testing service. You don't have all the usual results analysis to do yourself (tho you might want to do a quick check from time to time to make sure it's still working as expected/intended), you pretty much simply forward them as-is to the overlay maintainer, giving them the test results they requested. They get to figure out where the problem is and adjust (or if it's a tree-only bug that just happened to popup in this test, duplicate it as such, and file the bug accordingly) as necessary. > Now let's say I add more one overlay That won't happen in this one-overlay-at-a-time scenario. Again, the tree remains the primary focus, since all non-tinderbox.unmask-listed packages are by definition in-tree. You're simply providing a very specific overlay-against-tree tinderbox testing service, giving the requesting maintainers a bit more flexibility there, in exchange for offloading some of the responsibility you'd normally have for bug pre- analysis, etc, to them. They get both the additional flexibility of an overlay test, and rawer results that they have to do more work to parse, in the same package deal, but the focus remains the tree. There's no possibility or complexity of multiple overlays at once. > And you're also failing to understand how the whole tinderbox works. I > suggest you read the two posts I wrote on the topic, which will form a > basis for the documentation of the tinderbox: > > http://goo.gl/SM9Rp http://goo.gl/SF0Dz FWIW I'm subscribed to your feed and gentoo-planet both, so saw those. I read/skimmed them, but some of it was beyond me for just reading, tho it'd definitely be useful if I were actually implementing a tinderbox here. So if I missed something obvious therein, and this idea isn't practical either, forgive the waste of time. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
