21.08.2013 17:38, Wyatt Epp пишет: > Fundamentally, I see this as a problem of tooling.
I think that no tool can cover all cases of checking that software WORKS. I mean - in generic, for all kinds of software. You can guarantee if it builds, if it follow some QA rules about CFLAGS/LDFLAGS/whatever, but there is only one 100% way to guarantee that software works - run and do something useful :-). Yeah, i know about testsuites, that can help in that case. Unfortunately: 1) they do not cover all cases, but that's not so important; 2) often they are badly broken; 3) not every program carries test suite. So, we stuck at automation in run-time checks right at the beginning :-). But yeah, we could automate build-time checks, surely. > > I'd like to point out something that jumped out to me as a red flag > earlier (not to pick on you specifically, Tom; this is just the > cleanest example I saw), and turn it into an example: > > On Wed, Aug 21, 2013 at 4:51 AM, Tom Wijsman <tom...@gentoo.org> wrote: >> >> Well, they are listed there; but it's quite some work to actually go >> through that list, that is, manually check the bugs of ~2000 packages >> as well as file a STABLEREQ bug, takes quite a while... >> > This right here is a real problem. Any time you're talking about > doing anything on this scale "manually", you've already lost the > battle. You need a tool to minimise the overhead of time and > cognitive load. What would that tool look like? Think about the > steps involved and how you can reduce them to only the parts that > absolutely require decisions on your part. > >>> At least in the areas I usually work, I have found a combination of >>> the automatic stabilisation requests and imlate have definitely cut >>> back on the bitrot. >> >> A single unimportant bug can prevent the automatic STABLEREQ bug from >> getting filed; as for imlate, not everyone seems to know that tool, not >> everyone seems to run it. Attention for some stabilizations is lost... >> > First off, why do developers not know about the tools? How can this > be addressed? For a start, I'd suggest making sure the tools are at > least mentioned in the docs. Somewhere other than the amd64 Arch > Testing guide from 2006. [1] That's the only concrete (i.e. actually > in the DOCS rather than the ML archives) documentation I've found so > far, and it only references the file, rather than the tool. Maybe in > the devmanual Tools Reference? [2] > Good point, i agree. > But, imlate is a good example of a tool that could ease the time cost > of grindy crap. You showed before that it can get an ordinary count > bounded on n days. That's handy, but only a little. Build out: > - How many of those stablereq bugs reference versions no longer in the > tree? Those can probably get closed. > - How many have newer STABLE versions in the tree in the same slot? > Probably fine to close those, too. > - Of the remaining, how many have patches or ebuilds attached? Those > may be solved problems waiting for closure; shortlist them. > - How many are packages with newer versions that have been in the tree > for >30 days? Consider changing the target version, then. > - How many have open blockers, and what are those blockers (w/link and > summary)? Scan for low-hanging fruit jumping out in that list. > - Get views by category; are there categories where updates are more > important? Things in @system, and things with security concerns > (stuff in net-*) should probably get higher priority; games... > probably less so. > - Are there bugs with certain keywords in the body that should raise > priority? Things like "security" or "overflow" might be good > candidates. > - Are there bugs with certain keywords in the body that indicate it'll > be really easy to decide? e.g. "trivial" or "minor" might turn up some > of those super-small version bumps that you pretty much know aren't > going to affect stability. > > These are just examples off the top of my head, and by no means > bulletproof, but these are in the class of improvements that have ROI > because they reduce a task that previously took developer time to one > that takes CPU time. CPU time is essentially free compared to the > value of dev time. That's what i called constuctive criticism. Congratulations :-) > And I'm not saying more recruiting shouldn't happen, but relying on it > is no better than hoping at $deity for bugs to close themselves. ;) > > Cheers, > Wyatt > > [0] Okay, maybe in the "glory days" when we were higher up on > Distrowatch and thing were really kicking. (I know, I know, "DW isn't > representative", but really? Sabayon is doing better than we are, > now?) > [1] http://www.gentoo.org/proj/en/base/amd64/tests/index.xml?part=1&chap=2 > [2] http://devmanual.gentoo.org/tools-reference/index.html > -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop-effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead
signature.asc
Description: OpenPGP digital signature
