ruby on rails actionmailer receive character encoding issues

Question

Rails 3.0.11
ruby 1.9.3dev (2011-09-23 revision 33323) [x86_64-linux]
mail gem v. 2.2.19
postfix v. 2.7.0

I pipe received emails to an actionmailer. This usually works well unless the received email is not correctly encoded.

I have tried multiple approaches but always end up with an invalid byte sequence in UTF-8 error.

This is how I attempt to properly encode / force a proper encoding in the actionmailer:

email.text_part.decoded.force_encoding('UTF-8').encode('UTF-8', undef: :replace, invalid: :replace, replace: "")

This is how I forward mails to my app (content of the /etc/aliases file as suggested by the ruby on rails guide)

theaddress: "|/usr/local/rvm/bin/ruby-1.9.3-rc1 /var/www/myapp/script/rails runner -e mailer 'Mymailer.receive(STDIN.read)'"

I have also tried piping the mail through iconv first

theaddress: "|iconv -t UTF-8 -c |/usr/local/rvm/bin/ruby-1.9.3-rc1 /var/www/myapp/script/rails runner -e mailer 'Mymailer.receive(STDIN.read)'"

I also tried using Encding::Converter

Encoding::Converter.new("UTF-8//IGNORE", "UTF-8")

Thanks for any insight and help!

The Solution

I had to add following to my environment file:

Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8

Also I removed any force_encoding and encoding methods. So following now works:

email.text_part.decoded

You should post your solution as an answer. – David Morales Jul 19 '12 at 12:25 — David Morales, Jul 19 '12 at 12:25

score 2 · Accepted Answer · answered Jul 13 '12 at 19:51

Are you sure the encoding of the incoming email is UTF-8? My solution was to grab the charset from the header.raw_source

/charset=(?<charset>[^\s]+)\s?/ =~ part.header.raw_source

and if that is not null, gsub out any quotes and try to encode that

charset.gsub!('"', '')
raw = part.body.to_s
raw.force_encoding(charset)
decoded = raw.encode("UTF-8")