Quoteing c1205 / er.uqam.ca, on Fri, May 28, 2004 at 10:01:43PM +0900:
> I was playing around with the RMail package and I was missing RFC-2047 
> support. I found the "module Rfc2047" in 
> <20031204151316.GC849@jupp%gmx.de>

Probably the one I wrote.

> but noticed the following:
> 
> In the regex to discover encoded words:
> 
> |   WORD = %r{=\?([!#$%&'*+-/0-9A-Z\\^\`a-z{|}~]+)\?([BbQq])\?([!->@-~]+)\?=} # :nodoc:
> 
> I had to change % to \% to run. Maybe it's just Cygwin.

Looks like you are using ruby1.8. There's lots of warnings, too. I'll
fix it sometime, or you can send me a patch? :-)

> The second thing is that the module doesn't correctly interpret the 
> "encoded-word - linear white space - encoded word" sequence, where 
> all the white space should be deleted.
> 
> So I added a regex to delete this whitespace before further processing:
> 
> > module Rfc2047
> > 
> >   WORD = %r{=\?([!#$\%&'*+-/0-9A-Z\\^\`a-z{|}~]+)\?([BbQq])\?([!->@-~]+)\?=} # :nodoc:
> >|   WORDSEQ = %r{(=\?[!#$\%&'*+-/0-9A-Z\\^\`a-z{|}~]+\?[BbQq]\?[!->@-~]+\?=)\s*(=\?[!#$\%&'*+-/0-9A-Z\\^\`a-z{|}~]+\?[BbQq]\?[!->@-~]+\?=)}

Two comments:

1 - I don't think this will work. It will fix:

  encoded-word - linear white space - encoded word

but not:

   encoded-word - linear white space - encoded word - linear white space -
   encoded word

I.e, it only does pairs, so I don't think it does what you want.

2 - it will trash your input argument, which is fairly undesireable

I think you could do the match with a regex by using some-kind of regex
operator that matched a WORD, but didn't consume it. See below, I don't
have time to test it thoroughly, just one test case, but maybe it will
work for you. If it does, I think I could rewrite the regex to do this
in a single sweep, though I don't see efficiency as a concern, we're
talking mail headers here, they aren't that big!

> I also observed that decoding of non-Western character sets (Win-1251
> to Big5) to UTF-8 didn't work. Does anybody already suspect why or do
> I have to track down the error further?

Which version of rfc2047.rb do you have? I'm at 1.4, and it has a fix
for this, I believe, see below.

Sam

# $Id: rfc2047.rb,v 1.4 2003/04/18 20:55:56 sam Exp $
#
# An implementation of RFC 2047 decoding.
#
# This module depends on the iconv library by Nobuyoshi Nakada, which I've 
# heard may be distributed as a standard part of Ruby 1.8. Many thanks to him
# for helping with building and using iconv.
#
# Thanks to "Josef 'Jupp' Schugt" <jupp / gmx.de> for pointing out an error with
# stateful character sets.
#
# Copyright (c) Sam Roberts <sroberts / uniserve.com> 2004
#
# This file is distributed under the same terms as Ruby.

require 'iconv'

module Rfc2047

  WORD = %r{=\?([!#$%&'*+-/0-9A-Z\\^\`a-z{|}~]+)\?([BbQq])\?([!->@-~]+)\?=} # :nodoc:
  WORDSEQ = %r{(#{WORD.source})\s+(?=#{WORD.source})}

  # Decodes a string, +from+, containing RFC 2047 encoded words into a target
  # character set, +target+. See iconv_open(3) for information on the
  # supported target encodings. If one of the encoded words cannot be
  # converted to the target encoding, it is left in its encoded form.
  def Rfc2047.decode_to(target, from)
    from = from.gsub(WORDSEQ, '\1')
    out = from.gsub(WORD) do
      |word|
      charset, encoding, text = $1, $2, $3
      
      # B64 or QP decode, as necessary:
      case encoding
        when 'b', 'B'
          #puts text
          text = text.unpack('m*')[0]
          #puts text.dump

        when 'q', 'Q'
          # RFC 2047 has a variant of quoted printable where a ' ' character
          # can be represented as an '_', rather than =32, so convert
          # any of these that we find before doing the QP decoding.
          text = text.tr("_", " ")
          text = text.unpack('M*')[0]

        # Don't need an else, because no other values can be matched in a
        # WORD.
      end

      # Convert:
      #
      # Remember - Iconv.open(to, from)!
      begin
        text = Iconv.iconv(target, charset, text).join
        #puts text.dump
      rescue Errno::EINVAL, Iconv::IllegalSequence
        # Replace with the entire matched encoded word, a NOOP.
        text = word
      end
    end
  end
end