--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: text/plain; charset=US-ASCII

David Douthitt writes:

...

> Well, the Ruby version used to do:
> 
>    LogFile.open.readlines.each { }
> 
> Now it does
> 
>    LogFile.open { |f| f.readlines.each { } }
> 

...

> This following ruby code was replaced entirely by (ksh):
> 
>    for sys in $*
>    do
>       egrep ":.. [^ ]* (in\.|)$sys" $MESSAGES > ${sys}
>    done
> 
> (MESSAGESar/log/messages)
> 
> And it FLIES!  Here is the ruby code:
         ^^^^^
Arrrggghhhhh!!!

> 
> #!/usr/bin/ruby

...

> systems  rray.new
> 
> class String
>    def systemName
>       self.scan("^... .. ..:..:.. ([^ ]*)")
>    end
> end
> 
> File.open("/var/log/messages").each { |line|
>    line.chomp!
> #  sys  ine.scan("^... .. ..:..:.. ([^ ]*)")
>    sys  ine.systemName
> 
>    systems  ystems | sys
> 
>    if not ARGV.include?(sys[0][0])
>       print("    ", line, "\n")
>    end
>    }

Of course you have guessed it! Somebody has to complain now. This
comparsion is totally unfair. And I assume you know this fact!!! :-(

In your example 'grep' will do nearly all of the true work. The ksh
will serve only as wrapper and call 'grep' once for every system you
have passed via command line.

The ruby program, however, will do real work. No external program
is helping. Of course, most often a C program will be faster than a
plain Ruby one. But this will not save you ;-)))

Do you remember? In the original thread I had said: "I strongly
believe that my app using scripting and C together would be nearly as
performant as his one coded in C/C++ only." :-)))))))))

Okay, here we go:


Following is your example. I had to change the regexp a little bit,
otherwise it wouldn't match my syslog entries:


--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="sysgrep.ksh"
Content-Transfer-Encoding: quoted-printable

#!/bin/sh

MESSAGES=/var/log/messages

for sys in $*
do
   egrep ":.. *[^ ]* (in\.|)?$sys" $MESSAGES > ${sys}
done

--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: text/plain; charset=US-ASCII



Okay. Now the files necessary for Ruby. First the extconf.rb file that
configure our extension;


--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="extconf.rb"
Content-Transfer-Encoding: quoted-printable

require "mkmf"
create_makefile "_sysgrep"

--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: text/plain; charset=US-ASCII



Then our Ruby program that do nearly the same as your ksh script
(means nothing ;-):


--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="sysgrep.rb"
Content-Transfer-Encoding: quoted-printable

#!/bin/env ruby

require "_sysgrep"

MESSAGES = "/var/log/messages"

for sys in ARGV
  grep 16, sys, MESSAGES
end

exit 0

--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: text/plain; charset=US-ASCII



Then the extension module, the C code. This file serves only as prove
of concept and is not bullet proof coded (in fact no any error
handling):


--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="_sysgrep.c"
Content-Transfer-Encoding: quoted-printable


#include <ruby.h>
#include <strings.h>


#define MAXLIN 3072


static VALUE grep(VALUE self, VALUE offset, VALUE host, VALUE infile)
{
   FILE* fpin = fopen(RSTRING(infile)->ptr, "r");
   FILE* fpout = fopen(RSTRING(host)->ptr,"w");
   char  line[MAXLIN];
   char* sys = RSTRING(host)->ptr;
   long  syslen = RSTRING(host)->len;
   char* syspos = line + NUM2LONG(offset);

   while (fgets(line, MAXLIN, fpin))
      if (strncmp(syspos, sys, syslen) == 0)
         fprintf(fpout, line);

   fclose(fpin);
   fclose(fpout);

   return Qnil;
}


void Init__sysgrep()
{
   rb_define_global_function("grep", grep, 3);
}

--Multipart_Tue_Mar_28_04:34:19_2000-1
Content-Type: text/plain; charset=US-ASCII



So now I have done the following:

------------------------------------------------
Script started on Tue Mar 28 04:12:24 2000
cle@qiao > ruby extconf.rb 
creating Makefile
cle@qiao > make
gcc -fPIC -I/usr/local/lib/ruby/1.5/i586-linux -g -O2 -fPIC  -I/usr/local/include    -c _sysgrep.c -o _sysgrep.o
gcc -shared   -o _sysgrep.so _sysgrep.o -lc 
cle@qiao > time sysgrep.ksh qiao qiao qiao qiao   # your example

real	0m4.859s
user	0m1.990s
sys	0m1.250s
cle@qiao > time sysgrep.rb qiao qiao qiao qiao    # my one

real    0m4.169s
user    0m2.070s
sys     0m1.080s                   
cle@qiao > exit

Script done on Tue Mar 28 04:13:13 2000
------------------------------------------------

As you can see, now Ruby is even faster than your ksh solution. Of
course this evidence is not fair too, but in confirmation with my
cited statement above! :-)

And it is not more unfair than yours ;-))))

[nice code deleted ...]

If you would try to do this all in ksh *without* help from grep and in
the same complexity as your fine Ruby script, I guess you will go mad!
And I have my doubts, if this *pure* ksh solution would be faster than
the Ruby's one ;-)

> Now aren't you glad you asked for code?  :-)

I am for sure. I love to see code written by others. I can learn from
anyone! :-)

\cle

-- 
Clemens Hintze  mailto: c.hintze / gmx.net

--Multipart_Tue_Mar_28_04:34:19_2000-1--