I would like to identify partial matching of a regular expression, for a 
stream of input, as described in the pcrepartial(3) manpage. Is this 
possible with ruby Regexp, or would I have to wrap (a piece of) pcre? 
(or implement my own regular expression engine, hah!)

As an aside, what I am really trying to do is write a lexer that works 
on stream input, and can decide whether any of the eligible tokens match 
before reading EOF (which may be a long, long way off both in bytes and 
time). If you can think of another approach (that still uses regexes) 
that'd work too.

Thanks

1. http://www.gammon.com.au/pcre/pcrepartial.html