ruby -e 'require "set" ; s = Set.new ; ARGF.each_line {|z| s.add?(z)
and puts(z) }' input > output

or even more verbose

ruby -e 'require "set" ; s = Set.new ; ARGF.each_line {|z| if s.add?(z)
then puts(z) end }' input > output

Tris