This has been written without checking prior replies - there may be overlap.
First off, good work on the new report addressing more matters however this should have been your original report at a minimum. Before I even start I will outright state that I hope that Entrust actually improves throughout this and while this comment will be cleaned up it reflects an ongoing opinion as the report is read. First looking at the letter I will only note this paragraph: "We are disappointed as this does not represent Entrust values and falls short of the standards we set for ourselves. We also want to make sure it is understood that none of these lapses have been malicious or done with ill-intent to make the internet less secure. As a global CA we must walk a tightrope in balancing the requirements of the root programs and subscriber needs, especially for critical infrastructure. In some cases, we did not strike the right balance." It does trouble me that compliance is seen as a balancing point against issuance for critical infrastructure. There has been a common talking point of Entrust's delayed revocation incidents of the concept of irresponsible revocation. I point it to Entrust that such a scenario only presents itself when a CA is culpable of irresponsible issuance. As I read through this I keep seeing a repeating pattern of changing the organizational structure and creating committees and board of cross-discipline personnel. While this is all good in theory, I am concerned that this is not addressing the actual root causes of internal decision making and that the outputs will be just the same with a different label on the team providing it. Before I delve into any minutiae of the report itself I do find it noteworthy that in incident #1890898 (Entrust: Failure to revoke OV TLS - CPS typographical (text placement) error)) <https://bugzilla.mozilla.org/show_bug.cgi?id=1890898> we have a functional example of the new cross-functional team evaluating compliance and coming to a decision. Now this could be a third unspecified team but given the report I presume this is the template going forward, for brevity I'll keep this to the broad strokes: 2024-04-11: Issue opens, mis-issuance confirmed but no intent to revoke. A long conversation ensues, nothing changes until a day before the June 7th report appears. 2024-06-06 <https://bugzilla.mozilla.org/show_bug.cgi?id=1890898#c28>: "We reviewed and consulted with independent external experts on this revised analysis, and based on this broader consultation, we now believe there was no mis-issuance and thus no need to revoke the affected certificates. A detailed analysis is below." Following that analysis Mozilla and Chrome's Root Programs give a different opinion. 2024-06-18 <https://bugzilla.mozilla.org/show_bug.cgi?id=1890898#c42>: "On this basis, we will treat this as a mis-issuance, and intend to complete revocation by end of day Saturday, June 22." 2024-06-19 <https://bugzilla.mozilla.org/show_bug.cgi?id=1890898#c49>: "On the last question, our position is that there was no mis-issuance—not that there was a mis-issuance and we decided not to revoke which is the situation that recommends discussion with affected root stores." Now, using the 06-06 opinion as a basis we have an example of this new cross-functional team. They reviewed the original incident and came to a conclusion that a) was not the same as Entrust in April, but crucially b) was not compatible with the viewpoints of the Root Programs who spoke up. I am of the strong belief of evaluating institutional changes not on their stated internal changes, but on their outputs. The decisions are all we will see, they are all that will matter in practice. I will not detail line by line but I do notice that some factual discrepancies in the original report have been addressed. It would be good to find out how those came to be in the first case. There are still outstanding ones that I already stated previously. >>Note: During our investigation of this issue, we noted that a subset of 1,975 EV certificates were also issued without the Entrust EV policy identifier (OID), based on our interpretation of the ballot update. >This is also a miscount, presumably due to the original figure being 1963 + 6 certs on a test site that are being double-counted. On reading further in 2.1.1 Entrust have outright stated they still stand by their incorrect analysis as previously noted in this reply. This speaks volumes as to the decisions that will occur going forward. Within 2.1.3 there is a mention of Entrust continuing to issue certificates and advocate their position, but I am seeing no reflection as to the root cause of what causes them to advocate for their incorrect positions to this day. Not a single line of 2.1.4 addresses this either. Oddly 2.2.3 does not mention that on April 3rd "The issue was escalated to our verification team for further investigation.". Instead it purports a subtly different timeline where nothing happened until the 15th. The April 4th issue as stated in the bugzilla timeline is also absent. It is at this point in the report that my original reply must have gotten lost as I still have outstanding issues. I am quoting my original reply below: >>2.3.4 Improvement Plan >>... >>Automate CPR form to collect all required information at the outset from the reporter rather than relying solely on email >This goes back to policy issues discussed for years now, see: >https://github.com/mozilla/pkipolicy/issues/98 >https://github.com/cabforum/servercert/issues/201 >https://bugzilla.mozilla.org/show_bug.cgi?id=1650234 Now, moving on. In 2.4.1 I am again mis-identified as a reporter of the EV cert issue. This does not factually matter but is amusing as the initial factual corrections show that part of my response was read and applied. The only significant change I can see in 2.5.1 is the insistence that the analysis Entrust performed on the mis-issuance not existing on the OV TLS Typo issue must still be correct. As previously stated, I do not see how this is compatible with multiple Root Programs stating otherwise. I am confused about 2.5.3 though, it is about delayed revocation but the RCA is focused on the technical issue in the original incidents. 2.5.4 contradicts itself from paragraph to paragraph. A commitment to revocation and replacement, and then statements that delays will be managed on a case-by-case basis. It is a bit troubling to see the conclusion state the following: "The mis-issuances we experienced were technical non-conformities and, had any one of them happened in isolation, they would not have resulted in us taking such a hard look at our program and finding the opportunities that we did." Regarding ACME, I previously stated this question and will repeat it now: Can you make any guarantees that ACME will be a requirement for subscribers going forward, and that they will not be charged extra for using these systems? Looking into 4.3 Appendix 3: Success Measures I won't address each individually. I am curious how you intend to get the WebTrust annual audit results to result in 0 qualifications in the space of a year. I would suggest an element for Communication is added to address how often a question has to be restated or followed up on due to a lack of clarity and transparency. Otherwise the list presents a minimal standard for any complying CA, if this is not kept by any CA it would be further cause for concern. Once again in evaluating against what was requested I am struck at how the systemic failures are not being addressed. We have commitments to committees and boards, but the decisions are what truly matter. There is no mention of what policies caused these initial issues and how they were not adhered to. The 2020 commitments are only highlighted due to every comment noting it specifically, no attempt seems to exist to evaluate against historical issues. On the 2020 commitments I am deeply troubled about this statement in particular: "Knowledge of 2020 commitments was similarly confined to a small number of business unit employees, without broader leadership team/organizational awareness." This should have came up in audits which cover incidents on bugzilla. What happened? Did the auditor only address this with the same small number of business unit employees and somehow no note of these commitments made it into any report that went further up the chain? What confidence can we have in any bugzilla-specific commitments outside of this report going forward? As a final note I will highlight this section: "As part of our response process to the Mozilla community, Entrust assigned a group of three senior leaders, as well as an external consultant, to review each incident to validate and expand root cause analysis." Can we please have a breakdown on Entrust's end of what their original opinion was at the start of each incident, and how these personnel would evaluate the situation if it were to happen today? I sincerely hope that #1890898 is not an example going forward. The point of incident reports and action items is to ensure things do not repeat, knowing that the decision-making process is repaired would be one small step. - Wayne On Tuesday, June 18, 2024 at 6:35:48 PM UTC+1 Amir Omidi (aaomidi) wrote: > I am not going to say with certainty that Entrust is definitely putting > Chrome over Mozilla. However, I hope they know that most Linux systems out > there use the Mozilla root store directly. > On Tuesday, June 18, 2024 at 1:12:19 PM UTC-4 Mike Shaver wrote: > >> On Tue, Jun 18, 2024 at 12:49 PM Walt <[email protected]> wrote: >> >>> I'd just like to point out that we now have a situation where Entrust is >>> in the position of seemingly valuing the opinion of other Root Programs >>> over Mozilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1890898#c42 >>> >>> In Comment #37, it was hinted at (and made slightly more explicit in >>> #39) that the opinion of the Mozilla RP is that the attempt to >>> re-characterize these certs was not going to be looked kindly upon, and >>> only once a Google RP member explicitly said that it was the Google RP >>> opinion that the certs remained mis-issued was any movement made on >>> re-confirming the mis-issuance and taking action to revoke them. >>> >>> Also, if we're in a position where Entrust is finally able to commit to >>> revoking certs within a 5 day period (setting aside that these certs >>> technically need a delayed revocation bug as the mis-issuance was known as >>> far back as 2024-04-10), why are other incidents not able to be resolved in >>> this amount of time? Is it because Google showed up? >>> >> >> We’ve seen this behaviour in other incidents as well, I believe including >> the cpsURI one that has turned into a magnet for evidence of poor operation >> and lack of transparency and responsiveness. I remarked on it in my initial >> snarky reply to the Entrust Report, in fact. >> >> From a realpolitik perspective their behaviour could indeed be rational, >> especially when the only tool root programs have is distrust. Firefox would >> suffer substantial market disadvantage if it stopped trusting Entrust >> certificates when other browsers didn’t. I think people generally >> underestimate how much Mozilla would be willing to take near-term pain to >> protect users, but it’s also possible that I am overestimating it. >> >> Related to that, I think Chrome’s root program representatives have >> generally been more willing to take a concrete position quickly, so Mozilla >> might be waiting for more explanation when Chrome decides that there’s no >> explanation that could suffice, or similar. The root programs tend to be in >> agreement more often than not (virtually always with Chrome and Mozilla, I >> would say, excepting some slightly different root store populations), so it >> may be somewhat irrelevant whose opinion spurs motion. >> >> Realpolitik analysis aside, I do agree that Entrust has created the >> impression that they care much more about Chrome’s opinion than Mozilla’s, >> which IMO might not be the best posture to take given that Mozilla and its >> community are the locus for the processing and evaluation of the incidents >> in question. >> >> Mike >> >> >> >> -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/f3509b5c-fdb0-4a03-b34b-627d33bbc7c0n%40mozilla.org.
