The original poster was worried about keeping large result sets in memory.
If the DB driver is caching the results, you're still using memory on the
client machine.

On Sat, Dec 08, 2001 at 01:20:46AM +0900, Darrin Thompson wrote:
> I don't think this is necessarily the case. In practice, most of the 
> time you let the DB server send you all the results at once, but the 
> client driver begins returning results to the app before it has received 
> the entire results set. This magic is handled by the DB driver. This way 
> you don't always have to store the entire row set in ram. This also 
> tends to be faster. Using a cursor like you describe is probably quite 
> flexible but would result in a lot of round trips to retrieve a lot of 
> results. And it would be really rough on the server it things get busy.
> 
> I think most any DB driver does this magic buffering out of the box. 
> It's probably up to the DBD writer to just make sure that it's possible 
> to grab a row at a time.
> 
> Darrin
> 
> Mark Hahn wrote:
> 
> >You need to use an SQL CURSOR to do that.  One could write code to make a
> >true stream using the CURSOR.
> >
> >----- Original Message -----
> >From: " JamesBritt" <james / jamesbritt.com>
> >To: "ruby-talk ML" <ruby-talk / ruby-lang.org>
> >Sent: Thursday, December 06, 2001 8:26 PM
> >Subject: [ruby-talk:27783] DBI and large result sets
> >
> >
> >>I'm starting to use Ruby DBI, and I'm wondering about its use when
> >>
> >processing
> >
> >>very large result sets.
> >>
> >>Given that DBI sits on top of various DB drivers, is there some way to use
> >>
> >DBI
> >
> >>to run a query and process each record set without necessarily having to
> >>
> >hold
> >
> >>the entire result set in memory?  Or is DBI too abstract to allow you to
> >>
> >make
> >
> >>any assumptions about that?
> >>
> >>DBI::StatementHandle has fetch_many, but I think it works on a local store
> >>
> >of
> >
> >>the entire result set (whereas I want it to pull the data from the
> >>
> >database
> >
> >>only as I request a number of rows).
> >>
> >>Basically I want the SQL query to behave like a stream, not a single,
> >>
> >huge,
> >
> >>chunk o' data, but not have to tie the code to a specific database (if
> >>possible).
> >>
> >>
> >>Thanks,
> >>
> >>
> >>James
> >>
> >>
> 
> 
> 

-- 
Alan Chen
Digikata LLC
alan / digikata.com
http://digikata.com