On Dec 27, 2010, at 19:03, Loren Segal wrote:
> On 12/27/2010 5:03 PM, Eric Hodel wrote:
>> I highly support having a unified documentation tool and style for =
Ruby, but I personally dislike an enforced template like doxygen =
requires.
>=20
> What do you mean by an "enforced template"?

If we're building a documentation standard for ruby we'll need an =
example of good documentation, a "template" which we will enforce when =
documentation is submitted.

> Doxygen (or an alternative @tag based syntax) doesn't "require" =
anything from you, you're free to add the information you choose, just =
as with any documentation. @tag syntaxes are merely an *equivalent* to =
the existing documentation. The requirements come from a style guide, =
and that's part of documentation process, not some tool. However, since =
the Ruby C API is already starting to make use of @tag syntax with =
Doxygen, there is little reason not to leverage this syntax as part of =
the unified style. Furthermore, YARD already supports this syntax. I'd =
have no problems with RDoc adding support, too. I don't know of any =
tools that can document the C end of things, unless RDoc plans on adding =
support for the C APIs (which I guess I would not object to).

It would not be difficult to add to RDoc.

>> I've read documentation generated by such tools many times and have =
always found them difficult to navigate as there is not enough context =
to understand the interaction between all the components of an API.
>=20
> What does RDoc do better, here? Again, this is not the fault of @tag =
based syntaxes-- it is the fault of the tools. For instance, doxygen has =
some rough edges in displaying documentation, but this is the fault of =
the tool, not the use of @tags. Keep in mind that I'm not proposing a =
tool, just a *unified documentation style*. RDoc could certainly =
implement this if you think it does the best job at displaying =
information to a user.

Good documentation is independent of the tools used to process it or the =
syntax used to write it.  I don't think a different tool can make better =
documentation without human intervention.  Some characteristics of good =
documentation are consistent wording, approachable writing, =
comprehensive coverage and useable examples.  I'm unsure of how a tool =
can help with anything other than comprehensive coverage.

>> If ruby adopts a documentation minimum standard I don't think an =
enforced template should be our only requirement.
>=20
> I agree. As mentioned, the goal is to supply a unified *style*, not =
just a "template". @tag based syntax is merely a part of this style,

We are in agreement here.

> since it's already being used in the C API, and there is no =
alternative to properly documenting the C end of things than @tag based =
tools. It therefore makes sense to continue in this vein and unify the =
documentation in this way.=20

Granted.

> However, it's important to realize that supplying a "bare minimum" =
guideline to a documenter does a lot to improve the quality of =
documentation. Obviously, manual checks would have to be done to make =
sure people are not just "checking off boxes", but having those tags =
there lets us know what to look for when reviewing documentation, and =
lets *them* know what they should be writing. This is an improvement =
over the current adhoc "just-write-something" methodology. Tags also =
give us the ability to implement tooling later, because the docs are =
much more easily parse-able.

A common feature of documentation in ruby core is to "tag" method =
parameters in ++, <tt></tt> or </code></code>.  Matching the "tagged" =
items in a comment to the method's arguments is trivial way to implement =
an enhanced coverage report without requiring @tags.

One of the greatest joys I've found in programming is the ability to =
have the computer do work for me.  I would prefer to have a tool that =
could figure out what I haven't documented, or documented well enough, =
with the fewest possible concessions.

>> I question merging YARD into Ruby and replacing RDoc.
>>=20
>> Why aren't extensions or enhancements to RDoc being made instead?
>=20
> YARD has fairly significant architectural differences that don't lend =
well to simply writing extensions. YARD is actually a superset of RDoc, =
and not the other way around, so a merge of functionality wouldn't =
really work. For instance, YARD is not only a "parse and generate" tool. =
It is also often used as a 2 phase: "parse and store", "generate from =
stored data" tool (as exemplified by the 'yard server' tool).

Sounds like ri -f html.  ri uses RDoc::RI::Store to load data from a =
stored documentation tree.

> RDoc does not have this phase, so much of the functionality YARD has =
could not be implemented easily.

Loading an RDoc::RI::Store and running it back through an HTML generator =
could be implemented very easily.  The HTML generator may need to be =
adapted to handle pre-parsed comments which would involve adding two =
lines.

> YARD's parsing architecture also allows much more leeway to =
customizing the parsing phase than RDoc's limited markup preprocessing =
APIs. And yes, these APIs are used in at least a few plugins I know of.
>=20
>> RDoc has several APIs for adding such extensions, including plugins =
allowing alternate generators and additional directives:
>>=20
>> http://rdoc.rubyforge.org/RDoc/Generator.html
>> http://rdoc.rubyforge.org/RDoc/RDoc.html
>> =
http://rdoc.rubyforge.org/RDoc/Markup/PreProcess.html#method-c-register
>>=20
>> (I do not know if the APIs are sufficient as I've never received any =
communication from anyone attempting to use them for this task.  I would =
love some feedback on this matter.)
>>=20
> It's great that RDoc has these APIs. I've never used any of them =
myself, so I can't comment on their usefulness (at first glance they =
look good). However, a few questions/notes:
>=20
> - How is support for new constructs (DSLs) added in Ruby code? This =
one isn't very specific to this discussion, since Ruby core code doesn't =
use any non-standard DSLs, but I've found that this is one of the most =
powerful YARD APIs.

I built a proof-of-concept parser for Rakefiles:

https://github.com/rdoc/rdoc-rake

Since I am still supporting Ruby 1.8.7 I haven't been able to drop the =
irb-based pure-ruby parser.

Additionally there's Hugh Sasse's perl/pod parser:

https://github.com/rdoc/rdoc-perl_pod

And a fortran parser.

All these parsers build an RDoc::CodeObject tree which the generator =
handles transparently.  rdoc-rake subclasses RDoc::Include, =
RDoc::NormalModule and RDoc::AnyMethod to have a better-looking =
implementation.

> - Are there any ways to hook into the information generated by =
directives *after* they've been declared? For instance, can you check =
for the existence of a specific directive inside of a custom Generator? =
Again, this kind of functionality is used extremely often in YARD, and =
is extremely important to outputting documentation for arbitrary =
@tag-style metadata.

There is no restriction.  If a directive is unhandled it's stored in the =
code object's metadata hash.

> - Can the preprocessing API handle blocks of text (ones including =
newlines)? Certain tags require this (@example, for instance).

Currently, no.  Calling a registered handler with a third parameter (the =
comment's text) could add this functionality.

>> Finally, I'm unsure why you've never emailed me or otherwise =
contacted me to propose adding extensions to RDoc to support YARD-style =
comments.  I am not opposed to an extension that would support it.
>=20
> This is the first time I've proposed creating a unified style. This =
feature came at the request of Yugui to have a formalized discussion of =
the proposal I made in the ruby-core list linked above. It originally =
had nothing to do with RDoc specifically (and still doesn't). I have no =
problem adding this support to RDoc (in fact I suggested it in the =
previous email). I don't care which tool is ultimately used for =
generation, be it RDoc or YARD. However, would RDoc be generating =
documentation for the C API?

It wouldn't be much work to add, perhaps a week to fully polish.  My =
boss says I can work on this in place of company business, so I'll give =
it a shot.=