In mail "[ruby-talk:12601] http page download question"
"Ian Marsman" <imarsman / webstove.com> wrote:
> I am writing a script to download webpages from a favourite radio program
> in order to grab the URLs ot MP3s of recent shows that I want to then pass
> on to XMMS to play. I am able to get the pages, but often I get an error
> that the connection has been reset by the peer, specifically:
>
> /usr/lib/ruby/1.6/net/protocol.rb:582:in `sysread': Connection reset by
> peer (Errno::ECONNRESET)
In this case what ruby is saying is all. connection is reset by
(http) server. This error is thrown on, e.g. server down.
Catching error and retrying is only solution, I think. You can also
fetch rest of entity instead of whole body like this:
http.get path, { 'Range' => 'bytes=1024-2047' }
this code get body[1024..2047].
Then whole of "resume" get is (not tested yet):
# File.unlink 'data' if File.file? 'data' # if you need
param = nil
read = 0
begin
Net::HTTP.start(host) do |http|
File.open( 'data', 'a' ) do |f|
http.get( path, param ) do |str|
f.write str
read += str.size
end
end
end
rescue Errno::ECONNRESET
param = { 'Range' => "bytes=#{read}-" }
retry
end
NOTE: Don't retry in start{} block because Net::HTTP object does not
clear internal state (it's a bug).
Minero Aoki