On Jul 19, 2006, at 7:03 PM, Adam Shelly wrote:

> On 7/19/06, Joe Van Dyk <joevandyk / gmail.com> wrote:
>> On 7/19/06, Just Another Victim of the Ambient Morality
>> <ihatespam / rogers.com> wrote:
>> >
>> >     Okay, so the site doesn't exist where I think it does and I get
>> > redirected... in my browser.  Net::HTTP doesn't get redirected,  
>> it simply
>> > fails.  _This_ behaviour doesn't match my browser.  Is there any  
>> way I can
>> > get redirected or find out where it wants to redirect me and go  
>> there,
>> > myself?
>>
>> You might want to check out the API docs for Net::HTTP.
>>
>
> Ok, so Net:HTTP doesn't fail, it just returns pretty low-level
> information.  If you want browser behavior for redirects, you have to
> roll it yourself.  I ran into something similar 2 days ago when I was
> trying to fetch pages which required basic authentication  (Which, I
> just learned, is the mechanism behind those urername & password login
> popups that the browser generates).  I did manage to roll my own,
> based on some examples from this mailing list.
> Then I had to write a wrapper around that because sometimes my URI's
> pointed to file:// instead of http:// - another thing the browser
> handles invisibly.
> Then there are those pages that use cookies to track logins...
>
> So my question is, is there an existing library out there that
> implements all these things a browser does?   Where get(uri).page
> returns the source of the same page I would see if I put the URI in
> firefox?

WWW::Mechanize is a programmatic web browser for ruby:
http://mechanize.rubyforge.org/
http://rubyforge.org/projects/mechanize


>
> -Adam
>