Exchange 2016 Server Rejecting Certificate as Expired

Question

We have emails sent using TLS from a Windows 2012R2 (non-domain joined) web server in our DMZ to our internal Exchange 2016 server (also running on Windows 2012R2). This was working fine up until about a month ago when they stopped coming through (we just noticed it now because the emails are very infrequent). I forced a test mail through and when I look at the Transport Role protocol logs I see the following:

2020-06-24 11:02:33.524,
  MAILSERVER\Client Frontend MAILSERVER,
  0102030405060708,
  6,
  192.168.1.44:587,
  192.168.2.3:64961,
  *,
  " CN=*.example.com CN=Sectigo RSA Domain Validation Secure Server CA, O=Sectigo Limited, L=Salford, S=Greater Manchester, C=GB     
    0102030405060708090A0B0C0D0E0F10
    0102030405060708090A0B0C0D0E0F1011121314
    2020-03-17T19:00:00.000Z 
    2021-03-18T18:59:59.000Z 
    *.example.com;example.com",
  Sending certificate Subject Issuer name Serial number Thumbprint Not before Not after Subject alternate names

2020-06-24 11:02:33.540,
  MAILSERVER\Client Frontend MAILSERVER,
  0102030405060708,
  7,
  192.168.1.44:587,
  192.168.2.3:64961,
  *,
  ,
  TLS negotiation failed with error CertExpired

You can see that the validity dates for the cert are from Mar 17,2020 to Mar 18,2021.

The client side shows the following error log:

SERVER -> CLIENT: 220 mailserver.example.com Microsoft ESMTP MAIL Service ready at Wed, 24 Jun 2020 11:02:32 -0500
CLIENT -> SERVER: EHLO www.example.com
250-SIZE 36700160
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-AUTH GSSAPI NTLM
250-8BITMIME
250-BINARYMIME
250 CHUNKING
CLIENT -> SERVER: STARTTLS
SERVER -> CLIENT: 220 2.0.0 SMTP server ready
Connection failed. Error #2: stream_socket_enable_crypto(): SSL operation failed with code 1. OpenSSL Error messages:
error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed [E:\...\class-smtp.php line 374]SMTP Error: Could not connect to SMTP host.
CLIENT -> SERVER: QUIT
SERVER -> CLIENT: SMTP ERROR: QUIT command failed: Connection: closedSMTP Error: Could not connect to SMTP host.

The event log on the mail server shows the following event:

A fatal alert was received from the remote endpoint. The TLS protocol defined fatal alert code is 45.
- System 
 - Provider 
     [ Name]  Schannel 
     [ Guid]  {1F678132-5938-4686-9FDC-C8FF68F15C85} 
   EventID 36887 
   Version 0 
   Level 2 
   Task 0 
   Opcode 0 
   Keywords 0x8000000000000000 
 - TimeCreated 
     [ SystemTime]  2020-06-24 11:02:33.540386500
   EventRecordID 417754 
   Correlation 
 - Execution 
     [ ProcessID]  484 
     [ ThreadID]  1552 
   Channel System 
   Computer mailserver.example.com 
 - Security 
     [ UserID]  S-1-5-18 
- EventData 
    AlertDesc 45 

But, again, this event just indicates an expired cert.

Any ideas as to why Exchange thinks the cert expired? I have checked the date/time on both machines and they are correct to the second. Thanks!

Answer 1

Your information doesn't show the chain cert(s) used -- try openssl s_client -connect $host:$port -servername $host -showcerts and run each of the resulting PEM blocks through openssl x509 -text, or if you prefer put them in (separate) files and doubleclick. (If OpenSSL 1.1.1 you no longer need -servername $host.)

Many Comodo^WSectigo CAs used a USERTrust-to-AddTrust bridge which expired May 30; see https://security.stackexchange.com/questions/232978/how-to-fully-view-cross-signed-certificate-signatures and https://stackoverflow.com/questions/62107431/curl-error-60-ssl-certificate-problem-certificate-has-expired and as linked from both of those https://support.sectigo.com/articles/Knowledge/Sectigo-AddTrust-External-CA-Root-Expiring-May-30-2020 . In particular the only transparency-logged cert for Sectigo RSA DV-server is https://crt.sh/?id=924467861 under USERTrust RSA which has four known certs listed at https://crt.sh/?caid=1167 so if you are using the AddTrust one #4860286 you can see its notAfter was 2020-05-30 -- just about a month ago.

Answer 2

It is exactly what joeqwerty has said above, have you checked SMTP service assigned to certificate or bound to connector?

You could run below command to check:

Get-ReceiveConnector | FL Identity,RemoteIPRanges,PermissionGroups,Auth*,TlsCertificateName

For more detail: Configuring the TLS Certificate Name for Exchange Server Receive Connectors

Answer 3

So @dave_thompson_085 had the right idea in that the issue was the AddTrust cert that was at the root of my cert chain had expired. After fixing the issue on the Exchange server (by recreating the PFX cert file without the AddTrust root and re-importing it) I was still getting the same error. I setup a second client and tried sending an email and found it worked correctly so I knew the issue had to be with the client machine. Using WireShark on the Exchange server, I observed the TLS traffic between the two machines and could see that the Exchange server would send the cert and its immediate parent cert (Sectigo CA cert) and then the client would reject it as expired (even though both certs were valid). I assumed that the client must be verifying the received certs and somehow finding they chained up to the expired AddTrust root. The emails were being sent by PHPMailer so I looked in the PHP settings and found it had a CA bundle that it uses (as PHP cannot use the Windows Certificate Store apparently). Looking in that CA bundle I found and removed the AddTrust cert and that corrected the problem.