On Sat, Feb 16, 2019 at 12:24 PM George Macon via dev-security-policy < [email protected]> wrote:
> Prompted by the update to the Incident Response guidance earlier this > week, I wondered how well CAs are doing at giving timely updates on the > CA Incident bugs. > > I put together a quick prototype [0] that would examine the bug > whiteboards and last-update timestamps and report those where the next > update date is in the past. You can see the results [1] if you are > curious. I examined a couple of the bugs identified to see what the > status was: Thanks! It’s very interesting to see your results! For what it’s worth, I don’t think this is a particularly valuable metric. It’s certainly one we (peers) have discussed, but it’s not one that is a way to measure the back and forth that goes on with CAs. In particular, the Next Update is only set when there is a concrete and committed plan - and more issues than most do not catch that, because there’s substantial discussion to get to that point, and CAs aren’t doing particularly well there. It also doesn’t capture when the Mozilla peers themselves are swamped by issues - for example, the 68 outstanding issues by CAs. While distrusting CAs is certainly a path to ensuring less risk of overload, it’s perhaps a bit more than CAs with outstanding incidents may feel is desirable. As you note, it presently involves a rather large amount of human interaction to process. One way that members in the community can help is to engage on these issues by asking questions if there is more information needed. I would ask that members refrain from sharing their own conclusions on the bugs - and certainly believe it may be inappropriate to try and answer questions if they are not the CA in question (as some participants are inclined to do) - but there’s huge benefit in encouraging more and meaningful details and highlighting similarities that may exist. > > 1) https://bugzilla.mozilla.org/show_bug.cgi?id=1523680 > > The whiteboard does not contain a Next Update. The last action was the > CA providing the incident report on 2019-02-01. If there are no > questions or comments about the incident, I think this bug can be closed. > > 2) https://bugzilla.mozilla.org/show_bug.cgi?id=1518560 > > The whiteboard Next Update was set to 2019-02-05 on 2019-01-22. The CA > provided an update on 2019-02-04. This is before the stated Next Update > but was more than a week ago. If there are no further questions or > comments about the incident, I think this bug can be closed. > > 3) https://bugzilla.mozilla.org/show_bug.cgi?id=1524143 > > The whiteboard does not contain a Next Update. The last action was an > email to the CA requesting an incident report on 2019-01-31. The CA is > not in compliance with the guidelines. > > 4) https://bugzilla.mozilla.org/show_bug.cgi?id=1495524 > > The whiteboard Next Update was set to 2019-01-01 on 2018-10-03. The CA > has not commented in the bug since then. The CA is not in compliance > with the guidelines. > > 5) https://bugzilla.mozilla.org/show_bug.cgi?id=1492006 > > The whiteboard Next Update was set to 2019-01-24 on 2019-01-17 following > a comment by the CA that they would provide an update by 2019-01-24. The > CA has not commented in the bug since then. The CA is not in compliance > with the guidelines. > > I had hoped that a tool could automatically identify when CAs are not in > compliance with the guidelines, but it looks like that determination > actually requires reading and understanding the bug history. Oh, well. > > -George Macon > > [0]: https://gitlab.com/gmacon/moz-ca-incident-update > [1]: https://gitlab.com/snippets/1822917 > _______________________________________________ > dev-security-policy mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-security-policy > _______________________________________________ dev-security-policy mailing list [email protected] https://lists.mozilla.org/listinfo/dev-security-policy
