Correct way to handle UTF-8 for outgoing email

Question

Anyone could explain what's the correct way to send this email on UTF-8 form? Destination received as human unreadable code.

Edit1: added more detail on the code which shows where uploaded_file variable come from. Edit2: Added last section of the code

import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
def upload(upload_file):
ftp = ftplib.FTP('ftp.domain.com')
ftp.login("user","pass")
f = open(upload_file,'rb')
ftp_server_response = ftp.storbinary('STOR %s' %upload_file, f)
ftp_server_response_msg = ftp_server_response.split("/", 5)[4]
f.close()
ftp.quit()
os.remove(upload_file)
uploaded_filename = os.path.basename(upload_file)
html = """\
<iframe src="https://example.com/embed/{file_id}/{uploaded_file}" scrolling="no" frameborder="0" width="700" height="430" allowfullscreen="true" webkitallowfullscreen="true" mozallowfullscreen="true"></iframe>
""".format(file_id=ftp_server_response_msg, uploaded_file=uploaded_filename)
From = 'email@domain.com'
Recipient  = 'email@domain.com'

# Credentials
username = 'user01@domain.com'
password = 'password'
server = smtplib.SMTP('smtp.domain.com:587')

email_msg = MIMEMultipart('alternative')
email_msg['Subject'] = os.path.basename(upload_file).rsplit(".", 1)[0]
email_msg['From'] = From
email_msg['To'] = Recipient
email_msg_part1 = MIMEText(html, 'html')
email_msg.attach(email_msg_part1)

server.ehlo()
server.starttls()
server.login(username,password)
server.sendmail(From, Recipient, email_msg.as_string())
server.quit()

if __name__ == "__main__":
pool = Pool(9)
tasks = []
for root, dirs, filenames in os.walk("/ext_hdd/download"):
    dirs[:] = [d for d in dirs if d not in exclude]
    for extension in file_extensions:
        for filename in fnmatch.filter(filenames, extension):
            match = os.path.join(root, filename)
            file_size = os.path.getsize(match)
            if file_size > 209715200:

                    tasks.append(pool.apply_async(upload, args=(match,)))
            else:
                    pass

for task in tasks:
    print task
    task.get()
pool.close()
pool.join()

Everything looks like ASCII so you should have encoding issues. Can you post an image of the received email? — Alastair McCormack, May 18 '15 at 20:19
Hi, since I am posting to blogger via email, I could only show you the send part: http://pastebin.com/zu2CBzWK — Thomas G. Lau, May 19 '15 at 09:51
I suspect your filename variables are not UTF-8 encoded. Where is `upload_file` variable created? — Alastair McCormack, May 19 '15 at 11:01
`upload_file` source is still not shown and you've added the ftp function but its not called in your code. — Alastair McCormack, May 19 '15 at 15:36
Sorry, added last section of the code, nothing major is missing here — Thomas G. Lau, May 19 '15 at 15:46
Sorry, you've still not shown where `upload_file` is defined. No matter. What operating system are you using? — Alastair McCormack, May 19 '15 at 16:04

score 0 · Answer 1 · answered May 19 '15 at 19:23

The quick answer might be because you haven't specified the encoding on MIMEText and the subject header isn't defined as UTF-8. Assuming all your strings are UTF-8 encoded, you should use:

email_msg_part1 = MIMEText(html, 'html', "utf-8")
email_msg['Subject'] = Header(os.path.basename(upload_file).rsplit(".", 1)[0], "utf-8")

If this does not work, however, then you should concentrate on the source of upload_file.

I presume that upload_file comes from a file listing. On Linux, filenames aren't neutrally encoded like they are on Windows or enforced on OS X. This means that you can create a file with a filename UTF-8 encoded, which will look corrupted to a program that is reading the filename as ISO-8859-15.

It's possible that the files in /ext_hdd/download do not have UTF-8 filenames. You are then passing this non-UTF-8 encoded string for use where UTF-8 encoded strings should be used.

To resolve this, you should try to use Unicode strings where possible and let the mime library encode as it wishes. To get Unicode strings you need decode encoded strings like filenames. An easy way to do this is to pass a Unicode string as the directory name to os.walk():

os.walk(u"/ext_hdd/download")

This will use Python's locale to decode filenames where possible. Where's it not possible to decode, it will return the encoded filename. You will then need force an encoding on the string. Let's assume the encoding is actually Windows-1252. Add this to you os.walk() code:

if isinstance(filname, str):
    filename = filename.decode("windows-1252")

Then when you call set the message parts as given at the top.

I applied first two lines to my code, error appears: UnicodeEncodeError: 'ascii' codec can't encode characters in position 15-19: ordinal not in range(128); How could I be sure what's the encoding type is in use, which in your example is windows-1252 ? Can you also include full code on os.walk() section? I am not sure how to implement this correctly. — Thomas G. Lau, May 20 '15 at 05:24

Correct way to handle UTF-8 for outgoing email

1 Answers1