On 10/04/2018 02:55 AM, Kristof Beyls via lldb-dev wrote:
> Hi all,
>
> I’d like to share a few thoughts and analysis results on the LLVM bug life
> cycle, especially the reporting/triaging part.
>
> As one of the few people creating llvm bugzilla accounts when people request
> an account, I started to have a feel that many reported bugs, especially by
> first-time reporters, never get any reply or feedback, let alone be acted on.
> If people go through the effort of requesting an account, and then reporting
> the bug, they show motivation to contribute to the project. However, if then
> they see zero return on their effort spent, even if it’s just a confirmation
> of the bug indeed being real or an explanation of what they thought to be a
> bug isn’t actually a bug, I fear as a community we disincentify a large
> number of potential long-term contributors.
>
> The above was all based on gut feel, so I tried to gather a bit more data to
> see if my feel was correct or not.
> I scraped the bugs in bugzilla and post-processed them a bit. Below is a
> chart showing, year by year, how long it takes for a reported bug to get any
> comment from anyone besides to original reporter. If the bug is still open
> and didn’t have any reaction after half a year the chart categorizes is as an
> “infinite” response time.
>
>
> It shows that in recent years the chance of never getting a response to a bug
> report has been increasing.
> For some bugs - e.g. an experienced LLVM developer records a
> not-that-important bug in bugzilla - that may be just fine.
> However, I assume that for people reporting a bug for the first time, the
> majority may look at least for confirmation that what they reported is
> actually a bug.
> The chart shows (blue bars) that about 50% of first-time bug reporters never
> get any reply.
>
> I also plotted which components get the most reported bugs that don’t get any
> reaction and remain open:
>
> The percentage at the top of the bars is the percentage of bugs against that
> component that never get any reaction. The bar height shows the absolute
> numbers.
>
>
> I hope that at the “Lifecycle of LLVM bug reports” BoF at the upcoming dev
> meeting in San Jose (https://llvmdev18.sched.com/event/H2T3, 17th of October,
> 10.30am), we can discuss what could be done to improve the experience for
> first-time reporters and also to reduce the number of bug reports that
> seemingly get ignored completely.
> By sending this email, I hope to trigger discussion before the BoF, both by
> attendees and non-attendees, so that we have a more fruitful outcome.
>
> At first sight, to me, it seems that the following actions would help:
>
> * Let’s introduce some form of “triaged” state in bugzilla, to represent
> that a bug report has been accepted as describing a real problem; able to be
> acted on (e.g. has a suitable reproducer); and not being a duplicate of
> another bug report. Looking at
> https://bugzilla.readthedocs.io/en/5.0/using/editing.html#life-cycle-of-a-bug,
> maybe the best way to achieve this would be for newly raised bugs to by
> default go to an “UNCONFIRMED” state instead of “NEW”? Moving the status to
> “NEW” or “CONFIRMED” would indicate the bug has been triaged.
> * Would it help to have one or multiple people per component that volunteer
> to triage new bugs?
> * With the majority of developers being part of a team working on a product
> based on LLVM, I would assume that it is in the interest of most that
> reported bugs at least get evaluated/triaged? What is stopping those
> developers to find the time to do some triaging? Would a better notification
> mechanism be useful to notify when new bugs on a specific component come in
> that you could triage? Maybe per component try to have a few people on the
> “default CC list”, which seems easy to set up as a bugzilla administrator.
I think adding relevant people to the "default CC list" is a really
good idea. I tend to get pretty good response rates on bugs
when I add people to the CC list. Even with old bugs that have
been dormant for a while.
-Tom
> * Should we get rid of the "new-bugs/new bugs” component if we won’t have
> people triaging them?
> * Should we have some description of what a reasonable triage of a bug
> looks like? If we write such a page, we could also use that page to describe
> what we think should get recorded when closing bugs.
>
>
> Thanks,
>
> Kristof
>
>
>
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
