Generators are a nifty feature for Python 2.2, but they are a solution to
Python problems... problems that Ruby doesn't have.

In Ruby, it is trivial to pass a closure that interacts with the scope that
created it to another function. Thus, callbacks in Ruby are sufficiently
useful to solve the problems that Python generators were designed to solve.

The example that I read somewhere on the Python site was the tokenizer
problem. The original approach to the tokenizer problem was to have a
function that took two parameters as input: the text source and a function
pointer to a callback that processed the tokens. The problem with this
approach was that the callback function, designed by the user, didn't have a
very good way of keeping state between calls. It could use globals, but that
would suck.

The other approach was to write a tokenizer object that kept its state and
had a "nextToken" function that looked for the next token in the stream and
returned it to the caller. This solution hoisted the state problem off on
the tokenizer instead of the user. Still a problem especially considering
the difficulty of unrolling very complex loops of logic.

For Python, where closures aren't that easy to use, the generator is the
ideal solution. This way, you write your tokenizer function as if were to
use a callback so you don't have to unroll your loop. However, your function
gets wrapped up in an iterator object, so the user uses it as if it were
implemented as a Tokenizer object with a "nextToken" function.

In Ruby, you could probably emulate this behavior directly, but you really
wouldn't need to. Instead, you would just write your tokenizer to use a
callback, and your callback method would be a local closure with local state
persistant between calls. Problem solved.

Anyway, that's the story. Generators are a whitespace friendly solution to a
few of the problems that Ruby's super-duper closure mechanism solves just
fine.

-Will Conant

-----Original Message-----
From: Ned Konz [mailto:ned / bike-nomad.com]
Sent: Monday, October 01, 2001 10:56 AM
To: ruby-talk / ruby-lang.org
Subject: [ruby-talk:21876] Re: Python generators


On Monday 01 October 2001 09:28 am, Henning von Rosen wrote:
> Can anyone say two sentences of how the Python generator/iterator concept
> maps onto Ruby?

I'm no Ruby expert, but it seems to me (please correct me if I'm wrong) (and

it's more than two sentences):

Any Ruby method call can have an associated block argument. Within the
called 
method, this block can be invoked with the yield() operator. Since Ruby 
blocks can be closures (that is, they capture variable bindings), this
allows 
them to maintain state for use as iterators or generators.

Beyond this basic support, there is a mixed-in module called Enumerable that

provides a number of collection utility methods (collect, detect, 
each_with_index, entries, find, find_all, grep, include?, map, member?, 
reject, and select) using a generator method called each(), which when
called 
with a block argument, passes each of the elements of the sequence to the 
block in turn.

     collection.each { |element|  ... }

So if you want to work with the Enumerable interface, just provide a 
generator called each() and mix in Enumerable.

-- 
Ned Konz
currently: Stanwood, WA
email:     ned / bike-nomad.com
homepage:  http://bike-nomad.com