Quoting B.Candler / pobox.com, on Sun, Apr 10, 2005 at 11:51:26PM +0900:
> On Sun, Apr 10, 2005 at 01:07:10AM +0900, Sam Roberts wrote:
> > Problems:
> > - String has multiple representations in ASN.1. To encode it, you need
> >   to choose the String type. Particularly for DER, this causes
> >   round-trip problems - you decode a TeletexString to ruby String, then
> >   reencode, it gets encoded as UTF8String, and now you have mangled the
> >   data. In particular, cryptographic signatures fail.
> 
> A solution would be to mark each attribute in the class with its ASN.1 type:
> e.g.
> 
> class Foo
>   attr_accessor :bar, :baz
> 
>   asn1_attr :bar, OpenSSL::ASN1::ISO64String
>   asn1_attr :baz, OpenSSL::ASN1::UTF8String
> end

> Serialising Foo to der will then tag @bar and @baz correctly.

You willalso have to take into account invalidly encoded DER, though,
unless you can really take the moral high-ground and refuse to interop
with invalid DER. It's quite common for implementations to neglect the
leading zero necessary to make INTEGER positive if the high bit is set,
for example. So, when you reencode (correctly) you don't have the same
input. There's a whole set of common errors like this.

> If there's a possibility that a single attribute will be one of multiple
> types, then it should be wrapped in an ASN.1 'choice'

ASN.1 choices aren't a "wrapping" in the sense that you see any wrapping
in the BER or DER encoding, not unless you tag, anyhow. When an ASN.1
choice appears, you literally encode whichever one you want. This is the
common case for strings, for example. ASN.1 to BER/DER is one-way, there
are numbers of places where you cannot infer the ASN.1 from the
encoding. Not necessarily a criticism, just an observation.

> > - Memory overhead goes through the roof, because ASN.1 is very verbose.
> >   This is a variation of what happened in the XML world. XML looks like
> >   a tree, so people write tree-based APIs. Fast and easy... then they
> >   get a large document, or try and figure out why their code is so slow,
> >   and end up having to change to SAX, or some other stream-based API.
> 
> Sure, although it depends on how complex your object tree is. The existence
> of stream APIs for XML doesn't mean that in-memory data structure APIs for
> XML are worthless, and I think the same applies to ASN.1

No, but they tend to be written now (rexml, that next gen java api whose
name I forget) in terms of an underlying stream decoder. If you can live
with the cost of memory, you go for tree, but you have an alternative.
This is the right approach, I think.

> I'm not intimately familiar with OpenSSL's ASN.1 routines, for example, but
> they do seem to be memory-based (ASN1_OBJECT_new, ASN1_OBJECT_free etc)

It runs only on desktop big-memory systems. Take a largish word
document, and encrypt then decrypt it with its PKCS#7 APIs.

Mostly openssl deals with keys and certs, these are (relatively) small.
Even so, the verbosity of ASN.1 is truly astounding.

> Incidentally, Ruby's ASN.1 library does appear to have a 'traverse' method
> which acts as a stream parser. You still need to build a suitable state
> machine for it to 'yield' each element to, of course.

Probably built on top of openssl's tree-base routines, so you pay the
memory cose, and the complexity cost. 

Anyhow, mostly I just wanted to say writing a stream-based BER/DER
decoder in ruby would be easy. Writing stream-base DER encoders is
impossible, unfortunately (the ouput size is encoded at the beginning,
they should have used CER more often, but its too late now), but
stream-based BER encoders are also easy.

> > Btw, DJBs name is a bad word in the mail community, and I'm deeply
> > suspicious of anybody who suggests that somehow ASN.1 is "simpler" than
> > the IETF text-based protocols. I've implemented both, and its not true.
> 
> Was it DJB who suggested this?

Sorry, didn't mean to imply he said this. He did suggest the "no
accents" security "fix". His name is a bad word for other reasons.

Sam