On Thu, Feb 26, 2009 at 7:04 PM, Nelson B Bolyard <nel...@bolyard.me> wrote: > PEM's only real value is that it allows data to be copied and pasted > into and out of text documents. The base64 content is no more > enlightening, and IMO is significantly less informative, than the > binary DER. PEM encoding adds a MINIMUM 33% overhead in amount of data > that must be sent. That's a 33% tax on all the bandwidth providers and > memory chips, just to allow copying into text documents. It also requires > an additional pass over all the data to translated it to/from PEM form, > because the binary form is always the one that is valuable to the software. > Use of PEM may be appropriate in text documents, but CRLs being fetched over > the wire are fundamentally NOT text documents.
Realistically, everything is fundamentally text in the Old Protocols -- you know, the pre-LDAP, pre-PKIX, and pre-SNMP ones. (LDAP is anything but lightweight -- but it's more lightweight than the Directory Access specification in the X.5xx series of documents. SNMP is nowhere near simple -- but it's the only other thing, besides integration with X.500-type Directories, which uses BER, DER, or even the concept of an OID. The fact that the IETF has had to create simplified integration technologies for these severely broken ITU standards is simply a marketing move on the IETF's part -- so that they can claim conformance and interoperability with ITU standard protocols. Ever read H.323? Why do you think the IETF created SIP? That protocol was so broken there was no way to save it.) > To make all users pay a 33% bandwidth tax, so that the occasional and rare > person who wants to actually look at the contents can avoid binary, is > silly. Further, you can find out what kind of document it is from the > MIME content type of the http response, so the PEM header is redundant. I don't seem to recall the MIME type being available to me as a user when using wget or curl. As well, once a datastream has been serialized to my disk, for whatever reason (including the "I need to use the tools available to me that can actually decode and understand it" reason which led me to save ComSign's root CRLs to my disk), all of that metadata is *lost*. We don't yet have filesystems which can hold all the metadata -- in fact, our webservers, when sending static content files, rely on the file extensions to determine what MIME type to use to send it. (Apache's mime.types file is specifically what I'm thinking of, unless a specific AddType directive is added in a .htaccess or in the main server config.) > The main point here is that, on the Internet, interoperability is all > achieved based on compliance to Standards. A standard that says > "There's one way to do this" is generally preferred in the IETF > (but not in the ITU, obviously) to a standard that says "there are > many ways to do this and you must implement them all to be interoperable." A base standard, plus extensions to the standard when the base standard falls short. As I recall, this was the entire reason why MIME was created to handle multimedia extensions to email, otherwise transferred via SMTP. This was the entire reason why ESMTP was created, as well. Realistically, interoperability is not achieved by dogmatic adherence to Standards. And, also realistically, common practice (at least for wildcards in certificates, as discussed in the other i18n thread) is documented in a friggin' INFORMATIONAL (i.e., by definition not-a-Standard) RFC. And that's causing problems. Interoperability is recognizing what you're going to be thrown, and rolling with it. Yes, you have to draw a line somewhere -- but insisting on not performing petty transformations (by 'petty' I mean "it doesn't change the substance, only the form") is... well, petty. > Many of the places where software products already permit a variety of > data formats to be used exist simply because, at the time that the > specification for that software was written, it failed to adequately > specify data formats, usually because it never occurred to the authors > that more than one format was likely to ever be used. It seemed "obvious" > which was the right format, so no comment about format was provided. This is, I think, part of the reason why the PKCS series was created. This is also why there are multiple protocols -- DIFFERENT protocols, all of which must be implemented for interoperability -- pending before the IETF for the performance of very specific duties related to certificate provision, trust anchor management, and so on. It seems to be the preference of the IETF to have a single protocol -- but in this case, the people who insist on multiple protocols are winning. (Primarily because almost all of them apparently come from the ITU school of thought.) And the people trying to take up the fight to prevent it are simply adding to the confusion, by proposing MORE ways and MORE data formats. > That is the reason for utterly absurd implementations like the one > described in > https://developer.mozilla.org/en/NSS_Certificate_Download_Specification > which accepts certificates in no less than 8 different formats of data. You know, in the medical field (and perhaps a bit less so in EDI B2B commerce), there's companies that make mint by providing products that transform data from one serialization format to another. This problem has plagued computing for a long time, and it's not going to go away overnight. And, really, Firefox just isn't the place to make the stand on it. ("It's not working, because Firefox is fucking up again. I have to manually accept all these certificates for these sites that I'm going to.") > I applaud the IETF for specifying the formats for CRLs in sufficient > detail that products need not carry multiple implementations in order to > find one that works. I lament products that promote the use of non-standard > formats, and that some so-called "authorities" are unaware of the > standards' requirements. The point that I was trying to make is: even the software that this particular CA is using (RSA Certificate Manager, I believe was what it was called) isn't creating the proper format when a CRL is manually-generated. I'm glad you lament that products that don't produce the precise output that the Holy Standard demands exist. I'm glad that I'm not the only one who thinks that if software (client software, such as FF) expects a certain interchange of data, the server computers (the mission-critical software, such as running on the CA) should only generate that data format. The fact that a CA management software doesn't is antithetical to the entire concept of interoperability -- if even one party to the communication insists on Precisely The Ceremony Dictated By The Holy Standard. Again: if "-----BEGIN X509 CRL-----" is not a valid DER encoding (I insisted on the answer specifically so that it would be acknowledged that it is not), regardless of whether it's text or binary, it's the server's purview to decide what kind of bandwidth tax to impose on themselves and their clients. The server has to pay for the bandwidth that it uses, and if a lot of people use it, the server operator should see it as a cost/benefit equation that behooves them to reduce the bandwidth they're using simply by decoding those PEM transformations of the data. But for a server admin trying to get set up? Lemme put it this way: I am not sure that Wireshark understands DER, so it's rather difficult to figure out which piece of data is being sent if it doesn't behave the way that it's supposed to during setup. I'd love for a CA http/LDAP/whatever-access-method software to have a global switch that says "use base64 encoding (debugging mode)", which could then be unchecked once things were figured out, and cause things to go back to untransformed DER. But, this only works if the clients also support the debugging mode and debugging formats. (We have already seen that Firefox PSM does NOT provide useful error messages in at least some circumstances.) This is necessary *even though* Firefox adheres to the standards -- because it's unreasonable to assume that a CA http server that's being debugged is going to recognize Firefox and magically recognize that it needs a specific format. So, to you, Nelson, I ask again: what is the problem with recognizing an invalid DER sequence, running it through the base64 decoder, and then trying to handle it again? It is a well-known, well-documented, and often-used format, and it would promote interoperability in a way that dogmatic adherence to the Holy Standard does not. To Mr. Claesson, I would suggest that you file a bug report with whoever wrote your software, that The Standard Insists on having a non-base64 version of the CRL, and that having a version copied to the clipboard that you paste into notepad just doesn't work without additional transformation. -Kyle H -- dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
