On Aug 2, 2006, at 10:15 AM, Aaron Reimann wrote:

> Thank you guys.  I have not tried all that has been suggested, but I
> got this code emailed to me:
>
> ###
> require 'rubygems'
> require 'mechanize'
>
> url="http://edge.i-hacked.com/250-working-proxies-for-safe-web- 
> access-from-work-or-school"
> agent = WWW::Mechanize.new
> page = agent.get(url)
>
> page.body.scan(/http:\/\/www\.([^"]+)/) do
>        p $1
> end
> ###
>
> I had to install the 'mechanize' gem, but it works...overall.  I have
> to figure out how to "write" the output into a text file.  but this is
> pretty cool.

Mechanize has a method to get all the links for a Page:

     require "rubygems"
     require "mechanize"

     url="http://edge.i-hacked.com/250-working-proxies-for-safe-web- 
access-from-work-or-school"
     links = WWW::Mechanize.new.get(url).links.map { |a| a.uri rescue  
nil }.flatten
     File.open('links.txt', 'w') { |f| f.puts(links) }

This saves all the relative links, however.

-- Daniel