On Mon, 21 Apr 2003, Markus Wichmann wrote:

> Hello all,
>
> though I have to admit that I'm a newbie to Ruby I'll try to devise my
> questions intelligently ;-)

i'm *not* a newbie and my questions are generally arcane and un-intelligent so
i wouldn't worry about that.

> I'd like to parse an Apache logfile at regular intervals, say every 15
> minutes. If that part of the logfile that has been added since the last
> parsing contains a certain String, let's say "blueice", I'd like to be
> sent an e-mail.
>
> This is my first real Ruby experience, but I thought I should rather
> start with something practical instead of something academic.
>
> How do I realize the regular-interval-parsing? How can my application
> keep in mind what part of the logfile has already been parsed and which
> part is new? How am I able to send an e-mail using Ruby?
>
> Any hints in any form are greatly appreciated!

i would use ruby's built database, PStore, to store the date of the last run.
that way, you can simply cron the job - or run it by hand - whenever you feel
like it and the program will know the date of the most recently processed line
and can start from there - here's a little example.  a couple of things

  * the methods are defined in a BEGIN block for clarity/readability only.
    although it also works fine i don't generally do this in real code.
  * no mailing is going on, but you can read about that in the net/smtp
    section of the pickaxe's 'Network and Web Libraries' section.  you can
    download the book from http://www.rubycentral.com, but i'd reccomend
    buying it straight away if you already haven't - one to support the
    authors, two to let book companies know their is an audience out there for
    ruby books, and three because it is a super book for getting started in
    ruby

----CUT----
#!/usr/bin/env ruby
require 'pstore'

ACCESS_LOG = ARGV[0] || "/usr/local/apache/logs/access_log"
PSTORE     = ARGV[1] || "./access_log.db"

previous = get_previous_date()
date = nil

File.open(ACCESS_LOG) do |access_log|
  access_log.each do |line|
    date = parsedate(line)
    process_line(line) if(date > previous)
  end
end

store_current_date(date)


BEGIN {
  DATE_RE    = %r"\s\[(\d+)/(\w+)/(\d+):(\d+):(\d+):(\d+)"o
  MONTH_MAP  = Hash[
    %r/^jan/io =>  1,
    %r/^feb/io =>  2,
    %r/^mar/io =>  3,
    %r/^apr/io =>  4,
    %r/^may/io =>  5,
    %r/^jun/io =>  6,
    %r/^jul/io =>  7,
    %r/^aug/io =>  8,
    %r/^sep/io =>  9,
    %r/^oct/io => 10,
    %r/^nov/io => 11,
    %r/^dec/io => 12,
  ]
  def parsedate(line, re = DATE_RE)
    match = re.match(line) or raise (ArgumentError, "LINE HAS NO DATE")
    dd, mm, yy, h, m, s = match.to_a[1..-1]
    n = nil
    MONTH_MAP.map{|mp,mn| mp.match(mm) and n = mn}
    n or raise (ArgumentError, "COULD NOT MAP MONTH TO MONTH NUMBER")
    return Time.mktime(yy, mm, dd, h, m, s)
  end
  def get_previous_date()
    date = nil
    pstore = PStore.new(PSTORE)
    pstore.transaction do
      date =
	begin
	  pstore[:date]
	rescue
	  Time.at(0)
	end
    end
    return date
  end
  def process_line(line)
    puts(line)
  end
  def store_current_date(date)
    pstore = PStore.new(PSTORE)
    pstore.transaction do
      pstore[:date] = date
    end
  end
}
----CUT----

-a

--
  ====================================
  | Ara Howard
  | NOAA Forecast Systems Laboratory
  | Information and Technology Services
  | Data Systems Group
  | R/FST 325 Broadway
  | Boulder, CO 80305-3328
  | Email: ara.t.howard / fsl.noaa.gov
  | Phone:  303-497-7238
  | Fax:    303-497-7259
  ====================================