From: "Rik Hemsley" <rik / kde.org>
>
> #if Todd Gillespie
>  
> > Would one rather spend some quality time learning to use regexes fully, or
> > spend an indefinite amount of time debugging your one-off parser?
> 
> Regexp guru ? Perhaps you can tell me how to extract tokens from a
> string, where a set of delimiter characters are provided, but you must
> also return quoted strings "..." and bracketed strings (...) as separate
> tokens. Quoted characters (\n, \a, etc) should simply be 'unquoted' (\n
> -> n, \a -> a, etc.)
> 
> No, I'm not just randomly leeching some regexp knowledge here. My first
> Ruby project is a port of my internet mail message (RFC822) parser,
> which is in C++, using STL (and no regexps, for portability.)

Hi, I'm somewhat of a regexp-enjoying fiend myself; but after
Randal Schwartz posted this horrendous thing last March:
http://www.ruby-talk.com/cgi-bin/scat.rb/ruby/ruby-talk/12815
I've been somewhat wary whenever someone mentions RFC822.  :-)

However if the tokenizer you describe above is truly just what 
you need, it sounds kinda like a one-liner.  Maybe :) What do
you mean by "a set of delimeter characters are provided." ?
Are they known beforehand, or variable with each invocation?
Can quotes be escaped \" within the quoted string?


Thanks,

Bill