Gary Wright wrote:
>> I suspect that the server is the bottleneck, not the client.

Zed Shaw wrote:
> Better yet, don't use a Ruby web server at all, and use another tool you
> trust (httperf or ab and curl) to determine a good baseline performance.
> Once you've got what *could* be done with net/http then you can run
> net/http and compare.

Hi Zed & Garry,

I neglected to mention in my post that I have already double checked 
that WEBrick is not the culprit. Fetching from WEBrick using curl is as 
fast as using TCPSocket:

$ time curl -O http://localhost:12000/ten-meg.bin
  % Total    % Received % Xferd  Average Speed   Time    Time     Time 
Current
                                 Dload  Upload   Total   Spent    Left 
Speed
100 10.0M  100 10.0M    0     0  21.9M      0 --:--:-- --:--:-- --:--:-- 
29.4M

real    0m0.466s
user    0m0.013s
sys     0m0.093s

So I *know* Ruby can shift the bits around fast enough, it's just that 
net/http isn't playing the game.

> Also, I'm working on a faster alternative to net/http in the RFuzz http
> client.  Stay tuned for that, but you can play with it right now:

I'd definitely like to check that out!

I just think that net/http's abysmal speed is somewhat anomalous. I am 
confident there is a simple explanation - maybe some tight loop in there 
doing something not particularly clever - but I haven't had the time as 
yet to dive right into net/http and find the reason. And I haven't had 
much success with Ruby profilers either. More suggestions welcome here!

I have gone ahead and changed the critical section of my code to use 
TCPSocket instead. That solved the HTTP GET problem, but I still 
struggle with PUTs:

file = File.open(resultFile, "r")
http = Net::HTTP.new(@uri.host, @uri.port)
http.put("/put/" + URI.escape(File.basename(resultFile)), file.read)

Now that's not real pleasant because it relies on snarfing the whole 
file into memory first. I would have liked to do something like:

http.put("/put/" + URI.escape(File.basename(resultFile))) do 
|datasocket|
    while file.eof? == false
        datasocket.write(file.read(4096))
    end
end

This is similar to what net/http offers in the case of HTTP GET, but of 
course it's broken because of the aforementioned speed concerns:

http.request_get("/#{file}") { |response|
    response.read_body { |segment|
        # in here, 400 KB/s max and > 70% CPU utilisation ...
        outputFile.write(segment)
   }
}

I still call myself a Ruby newbie, so one of my concerns is that perhaps 
I'm Just Not Getting It and that if I followed the Ruby Way my troubles 
would vanish :)

-- 
Posted via http://www.ruby-forum.com/.