rubylexer version 0.7.4 has been released! * <http://github.com/coatl/rubylexer/> * <http://rubyforge.org/projects/rubylexer/> RubyLexer is a lexer library for Ruby, written in Ruby. Rubylexer is meant as a lexer for Ruby that's complete and correct; all legal Ruby code should be lexed correctly by RubyLexer as well. Just enough parsing capability is included to give RubyLexer enough context to tokenize correctly in all cases. (This turned out to be more parsing than I had thought or wanted to take on at first.) RubyLexer handles the hard things like complicated strings, the ambiguous nature of some punctuation characters and keywords in ruby, and distinguishing methods and local variables. Changes: ### 0.7.4/5-20-2009 * 2 Major Enhancements: * preliminary support for ruby 1.9 * utf8 inputs should now work... more or less * 5 Bugfixes: * better detection of illegal escapes and interpolations in strings * indicate error on unterminated here body * fixed pattern of keywords that can't start a param list (ignores ?,! now) * in is_var_name?, check for global/instance vars first * comma and star in a true lhs should be correctly marked as such, now * 2 Minor Enhancements: * added tag field to Token; I hope many flags can be coalesced into tag. * note line that all strings (and here docs) start and end on * <http://github.com/coatl/rubylexer/> * <http://rubyforge.org/projects/rubylexer/>