Office 365 - EWS Managed API 2.0 Impersonation 401 unauthorized/503 not available

Question

private ExchangeService connectToEWS(string email, bool impersonate = false) {
   WebCredentials credentials = new WebCredentials(username, password, domain);
   ExchangeVersion version = (ExchangeVersion)exchangeVersion;
   ExchangeService service = new ExchangeService(version) {
      Credentials = credentials,
      Url = new Uri("https://outlook.office365.com/ews/exchange.asmx")
   };
   if (impersonate) {
      service.HttpHeaders.Add("X-AnchorMailbox", email);
      service.ImpersonatedUserId = new ImpersonatedUserId(ConnectingIdType.SmtpAddress, email);                      service.CookieContainer = new CookieContainer();
   }
   service.Timeout = int.MaxValue;
}

Using the code above to create an ExchangeSession and

ExchangeService service = connectToEWS(email, true);
FolderId msgFolderRoot = WellKnownFolderName.MsgFolderRoot;
FolderView view = new FolderView(int.MaxValue);
view.PropertySet = PropertySet.IdOnly;
var result = service.FindFolders(msgFolderRoot, view);

I receive a 401 unauthorized exception on the FindFolders call. If I use the other constructor of WebCredentials that takes only the smtp and password, I get a 503 server not available exception. When I do not use impersonation (delegate access instead), no exceptions are thrown and I can get the list of folders however I need to be able to support impersonation.

I have added ApplicationImpersonation role to the service account (username in the first code block). Is there another role required for impersonation? I can not find anything in the EWS impersonation documentation. Every other thread about this issue points to the UPN usage rather than smtp but I am fairly sure this smtp is the same as the UPN. I don't actually know where to look within o365 however.