Tony Arcieri wrote:
> On Thu, Feb 26, 2009 at 12:03 AM, Mike Gold <mike.gold.4433 / gmail.com>wrote:
>> Incidentally, 1.8 is odd because it is one big AST in C, with no
>> compilation.  Obviously 1.8 is very slow because of this.  ParseTree
>> is just giving back what is already there.  ParseTree is not available
>> in 1.9 or JRuby, nor is there a plan for it (and due to the nature of
>> compilation and optimization, I don't think it will happen).  MRI is a
>> prototype, implementation-wise.
> Yes, you are correct, and I really wish ruby-core would embrace ParseTree
> and make it a standard feature of the language.  The fact they haven't has
> precluded alternative implementations like JRuby from implementing it.

Actually, I believe it is the other way round: embracing ParseTree
would preclude alternative implementations like JRuby from being
implemented in the first place.

Well, I admit, I was exaggerating. However, until recently, JRuby
*had* a (somewhat) ParseTree-compatible AST. The reason why they
decided to drop ParseTree-compatibility was simply that it would be
impossible to evolve the compiler while keeping the AST compatible.

ParseTree is intimately tied to the actual layout of the AST inside
MRI and to the actual inner workings of MRI. That would mean that the
alternative implementations would be intimately tied to the inner
workings of MRI -- in other words, they wouldn't actually be
"alternative" implementations any more, just clones.

This is especially bad, since MRI is actually a pretty marginal edge
case: it only supports Ruby 1.8, most other implementations support
either Ruby 1.9 or Ruby 1.8 *and* 1.9 in the same parser. MRI is the
only pure interpreter, *every* other implementation (and there's about
a dozen of them) has a compiler -- some even have *only* a compiler
and no interpreter at all. And, last but not least, even MRI's creator
doesn't consider it a very good Ruby Implementation.

JRuby is actually a pretty extreme countercase: where MRI's parser and
AST are doing exactly one thing, JRuby's parser and AST are used to
implement an interpreter, a dynamic JIT and a static ahead-of-time
batch compiler within the JRuby project, plus an interactive resident
background parser for syntax checking, syntax highlighting, code
formatting, code generation, code completion, static analysis, type
inference, code transformation and refactoring in at least 3 different
IDEs, and even to implement a compiler and type inference engine for
*another* language implementation (Duby).

> There is JParseTree for JRuby but it isn't actively maintained.  And of
> course YARV practically precludes a ParseTree implementation so there's
> little motivation for alternative implementations to even try to support
> it.  The future of Ruby seems to be one without ParseTree.
> 
> So Lisp 1, Ruby 0 I guess.

I wouldn't say that. There are better ways than ParseTree. A
standardized generic, abstract, high-level parser API for example.

Countless man-hours and dollars, and even entire companies have been
lost in the battle for implementation-independent ASTs. There is a
reason why JVM bytecode and not ANDF rules the world. Let's learn from
those mistakes.

jwm