On 10/04/2018 02:55 AM, Kristof Beyls via lldb-dev wrote: > Hi all, > > I’d like to share a few thoughts and analysis results on the LLVM bug life > cycle, especially the reporting/triaging part. > > As one of the few people creating llvm bugzilla accounts when people request > an account, I started to have a feel that many reported bugs, especially by > first-time reporters, never get any reply or feedback, let alone be acted on. > If people go through the effort of requesting an account, and then reporting > the bug, they show motivation to contribute to the project. However, if then > they see zero return on their effort spent, even if it’s just a confirmation > of the bug indeed being real or an explanation of what they thought to be a > bug isn’t actually a bug, I fear as a community we disincentify a large > number of potential long-term contributors. > > The above was all based on gut feel, so I tried to gather a bit more data to > see if my feel was correct or not. > I scraped the bugs in bugzilla and post-processed them a bit. Below is a > chart showing, year by year, how long it takes for a reported bug to get any > comment from anyone besides to original reporter. If the bug is still open > and didn’t have any reaction after half a year the chart categorizes is as an > “infinite” response time. > > > It shows that in recent years the chance of never getting a response to a bug > report has been increasing. > For some bugs - e.g. an experienced LLVM developer records a > not-that-important bug in bugzilla - that may be just fine. > However, I assume that for people reporting a bug for the first time, the > majority may look at least for confirmation that what they reported is > actually a bug. > The chart shows (blue bars) that about 50% of first-time bug reporters never > get any reply. > > I also plotted which components get the most reported bugs that don’t get any > reaction and remain open: > > The percentage at the top of the bars is the percentage of bugs against that > component that never get any reaction. The bar height shows the absolute > numbers. > > > I hope that at the “Lifecycle of LLVM bug reports” BoF at the upcoming dev > meeting in San Jose (https://llvmdev18.sched.com/event/H2T3, 17th of October, > 10.30am), we can discuss what could be done to improve the experience for > first-time reporters and also to reduce the number of bug reports that > seemingly get ignored completely. > By sending this email, I hope to trigger discussion before the BoF, both by > attendees and non-attendees, so that we have a more fruitful outcome. > > At first sight, to me, it seems that the following actions would help: > > * Let’s introduce some form of “triaged” state in bugzilla, to represent > that a bug report has been accepted as describing a real problem; able to be > acted on (e.g. has a suitable reproducer); and not being a duplicate of > another bug report. Looking at > https://bugzilla.readthedocs.io/en/5.0/using/editing.html#life-cycle-of-a-bug, > maybe the best way to achieve this would be for newly raised bugs to by > default go to an “UNCONFIRMED” state instead of “NEW”? Moving the status to > “NEW” or “CONFIRMED” would indicate the bug has been triaged. > * Would it help to have one or multiple people per component that volunteer > to triage new bugs? > * With the majority of developers being part of a team working on a product > based on LLVM, I would assume that it is in the interest of most that > reported bugs at least get evaluated/triaged? What is stopping those > developers to find the time to do some triaging? Would a better notification > mechanism be useful to notify when new bugs on a specific component come in > that you could triage? Maybe per component try to have a few people on the > “default CC list”, which seems easy to set up as a bugzilla administrator.
I think adding relevant people to the "default CC list" is a really good idea. I tend to get pretty good response rates on bugs when I add people to the CC list. Even with old bugs that have been dormant for a while. -Tom > * Should we get rid of the "new-bugs/new bugs” component if we won’t have > people triaging them? > * Should we have some description of what a reasonable triage of a bug > looks like? If we write such a page, we could also use that page to describe > what we think should get recorded when closing bugs. > > > Thanks, > > Kristof > > > > _______________________________________________ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
