Chad;

This might be a stupid question, but it's always worth asking just in
case =).  Are you sure that the performance problem in the code above
is in fetching the document.  A 150M xml file can take a long time to
parse into a REXML document.

Actually, in a quick read of this:

>   if(response.content_type == "text/xml")
>       doc = REXML::Document.new(response.body)
>       #check to see if Metacat is sending an error message or EML
>       if(doc.root.name == 'error')
>         nil
>       else
>         Eml.new(response.body)
>       end
>     elsif(response.content_type == "text/plain")
>       DataTable.new(docid, response.body)
>     end
>   end

It doesn't look like your using 'doc' to do anything except check the
root node.  Meanwhile REXML has to parse the entire document--tree
parser.  You might want to give one of the streaming parsers a shot.


-- 
Lou.