1gor wrote:
> I need  Ruby to access a set of external command line applications for
> dataseries analysis. Each of these little apps simply reads data from
> stdin and then outputs processed results to stdout (using scanf and
> printf, they are all written in C).
> 
> These functions have to be accessed repeatedly when iterating over
> large chunks of data, so accessing them through the pipe (popen or
> popen3) doesn't work.

Do they need changing arguments?  Otherwise you could just start an 
instance of each via popen and then reuse that throughout the whole 
program.  Or do these programs detect end of input data via EOF on STDIN?

> Launching/closing an external process 3000 times
> in a row results in errors. And that's not efficient.

Yeah, true.

> So I have to
> write extension for each of those little C programs, a wrapper of
> sorts.
> 
> I have learned how to write and compile an extension for Ruby. I can
> import an extension, pass an array of data to it and return some data
> from the extension. The problem is that I have no C experience and
> dread thinking of having to replace each 'scanf' routine with my custom
> array of imported data (memory allocation, writing iterator etc.). Each
> input function here starts with:
> 
>      while (scanf("%lf", &y) == 1) { ... etc.
> 
> and then follows to process input data, handle cases with input buffer
> is too large etc. I would very much like to leave these input()
> functions as they are and just feed them my data from Ruby code.
> 
> Is it possible to fill stdin buffer with data from Ruby code and then
> simply call existing C function that uses 'scanf'? I couldn't simply
> assign an array to $stdin variable in Ruby, gave TypeError (File
> expected)...

Hm, I'm not a seasoned C programmer either, so I cannot give you a 
definitive answer to this.  However, I doubt that you can simply inject 
stuff into stdin.

Here are two more ideas:

1. Rewrite your programs to act as if they saw EOF from stdin when they 
receive a certain signal (say SIGUSR1).  That way you could use a single 
  process throughout your whole Ruby script with popen and just need to 
send the signal at certain points in time.  You could even change 
behavior to use the first line read from the pipe / stdin as command 
line arguments.

2. If it's feasible rewrite the processing in Ruby.  I guess you want to 
use a C program for certain reasons - these might be performance, not 
having to rewrite the code or whatever.  If it's not performance then 
maybe you can replace the stuff with 10% LOC in Ruby.

Kind regards

	robert