On 5/30/09, Clifford Heath <no / spam.please.net> wrote:
>> This is very interesting, I could use this in RubyMacros as well. But
>> then that brings up another issue; what if I want to use both at once?
>
> I think you'd need to create a custom loader that knows how to load the
> file having combined syntax.

Typically, yes. And that's the biggest challenge. (If you're lucky,
you might be able to make earlier stages of the pipeline tolerant
enough to just pass through new syntax which they don't understand.)

> When Nathan Sobo created Treetop, one of his hopes was that Ruby
> could eventually incorporate PEG parsing technology and add his "rule"
> keyword. Because PEG parsers are composable, this would allow you to
> define new parse rules within a Ruby program, which would integrate
> fully into the existing Ruby grammar, or be used to introduce new
> sub-languages that mesh in nicely... My take on that was then to

That's cool, even if I have no idea how this composable grammar would
work. I have similar goals with RubyMacros, which right now is still
very far from that level of flexibility. I actually
hope macros will be a somewhat more approachable interface to the same idea.

> extend Kernel.require so that each file extension used a different
> top-level parse rule *from within the combined grammar*, and you have
> a full meta-language framework... That's kinda what Polyglot is about.
>
> It's kinda what you're talking about below, but done "properly".

I can see the attraction of not needing to parse and unparse things
all the time.

> It's certainly possible. How would one loader pass output to the next?
> I wouldn't want to create the intermediate content as files.

Maybe a pipe? I suggest it reluctantly since I don't think my own
tools can use a pipe. Ruby is complicated, and it's nice to be able to
seek back and forth in the input stream. I don't see what's wrong with
a temp file; it's better than a string. Or you can define some kind of
api for connecting preprocessors, but I can't see the advantage of it
over one of those three possibilities.