non-ascii email addresses and SMTP

Question

Is there any way that this email will ever function? łōręmıpśum@łōręmıpśum.com This is not a totally frivolous pursuit. We have Polish users who have input non-ascii emails into our app.

The gmail validator flags it as invalid, but maybe it's just such a rare case that they haven't allowed for it. The punycode for "łōręmıpśum.com" is "xn--rmpum-j0a3o6e4dtg.com" and info@xn--rmpum-j0a3o6e4dtg.com seems to work fine. The user-part is giving me a headache though.

I had assumed I could utf8+percent encode it as I would in a url (which comes out to "%C5%82%C5%8Dr%C4%99m%C4%B1p%C5%9Bum"), but the SMTP servers return my mail with an error:

The mail system

<%C5%82%C5%8Dr%C4%99m%C4%B1p%C5%9Bum@xn--rmpum-j0a3o6e4dtg.com>: host
    eforward3.registrar-servers.com[38.101.213.199] said: 554 5.7.1
    <%C5%82%C5%8Dr%C4%99m%C4%B1p%C5%9Bum@xn--rmpum-j0a3o6e4dtg.com>: Relay
    access denied (in reply to RCPT TO command)

Final-Recipient: rfc822; %C5%82%C5%8Dr%C4%99m%C4%B1p%C5%9Bum@xn--rmpum-j0a3o6e4dtg.com
Original-Recipient: rfc822;%C5%82%C5%8Dr%C4%99m%C4%B1p%C5%9Bum@xn--rmpum-j0a3o6e4dtg.com
Action: failed
Status: 5.7.1
Remote-MTA: dns; eforward3.registrar-servers.com
Diagnostic-Code: smtp; 554 5.7.1
    <%C5%82%C5%8Dr%C4%99m%C4%B1p%C5%9Bum@xn--rmpum-j0a3o6e4dtg.com>: Relay
    access denied

It seems like the email passes through the local SMTP server fine, but fails at the remote end. Possibly the namecheap mail server is configured in an ASCII-centric manner.

My chief question: I've already proven to myself that the server-part of the email can be be Polish, but can the user-part? If not, why? It seems to me that the above email, while incredibly ugly, is just ASCII, and shouldn't be treated specially by email systems that are not unicode-aware.

Answer 1

Unicode is a total red herring; this has nothing to do with weird characters or encodings and everything to do with %.

"Relay access denied" is an error message from Postfix, produced when you're asking it to forward your mail to some other server. ("Open relays" were, and likely still are, a major contributor to the spam problem in much the same way that open proxies make it hard to ban a miscreant.)

So, why does Postfix think you're asking it to relay this mail? Let's check the docs:

2.3.3.1. Open Relay Prevention

Do not relay mail from remote hosts to remote addresses! (Unless the sender is authenticated).

This may seem obvious to most of us, but apparently this is a frequently overlooked consideration. Also, not everyone may have a full grasp of the various internet standards related to e-mail addresses and delivery paths (consider "percent hack domains", "bang (!) paths", etc).

Hang on, what? I don't think I have a full grasp of whatever those last two things are.

A googling explains:

Years later, during the creation of the CSNET, another kind of relaying notation was developed. The first instantiation of the CSNET established computers at RAND and the University of Delaware to provide email connectivity for sites not connected to the ARPANET. In order to indicate the required relaying, a notation called the "percent hack" was created, so that an email address could be specified in the following form:

mary%compsci@udel

The above email address specifies that the user "mary" has an account on the computer "compsci", and can be reached through the relaying host at the University of Delaware. The % sign was chosen because it wasn't used by any of the other popular email systems, and because it was visually similar to the notation "C/O" used to designate "in care of" for letters sent with postal mail. Although the percent hack enabled multilevel relaying, its actual use was limited to a single relay -- one % and one @.

The same page also explains the hacky use of !.

I don't think either of these behaviors are specced (hence, "hack"...); both % and ! are supposed to be plain characters in a local-part. But Postfix supports both of these hacks by default, so they're de facto reserved.

---

As for the Unicode: percent-encoding is a URI thing only and there's no reason to expect it would work anywhere in an email address.

RFC6531 extends SMTP to "international" characters. It's about as clear as any other email-related RFC, but it sounds like you can use UTF-8 in a local-part, and either the receiving server will understand it via the SMTPUTF8 extension or... it won't. (Presumably, anyone with a Unicode local-part is already using an email server that understands Unicode.)

Answer 2

The user-part just needs to be utf8 encoded, without the % encoding. This email totally works "łōręmıpśum@xn--rmpum-j0a3o6e4dtg.com" with utf8 on the left and punycode (aka IDNA) on the right.