I posted this on askubuntu.com yesterday but haven't gotten any responses.
We run a production Ubuntu server that serves thousands of people per day. Sendmail is currently not working on this server. We've spent days trying to get it back up but we haven't found any solution yet.
There's currently a bug report that appears to be related to this problem, so it affects more people than just us.
Here's what we know.
On Sunday we ran updates on the server. The next day we found that sendmail was failing to send emails.
/var/log/sendmail.log reports "stat=Deferred" on every email entry.
It also occasionally repeats the following message:
STARTTLS=client, error:connect failed=-1, SSL_error=1, errno=0, retry=-1
STARTTLS=client: 7042:error:14082174:SSL routines:SSL3_CHECK_CERT_AND_ALGORITHM:dh key too small:s3_clnt.c:3338:ruleset=tls_server, arg1=SOFTWARE, relay=xxx.xxx.edu, reject=403 4.7.0 TLS handshake failed.
We checked the logs on the SMTP server and found the following:
06-25T10:57:20-06:00 gw26 sm-mta[17229]: STARTTLS=server, error: accept failed=0, SSL_error=1, errno=0, retry=-1
No explanation available 2015-06-25T10:57:20-06:00 gw26 sm-mta[17229]: STARTTLS=server: 17229:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1110:SSL alert number 40
Explain this log line 2015-06-25T10:57:20-06:00 gw26 sm-mta[17229]: t5PGvKk0017229: opus.byu.edu [128.187.102.135] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
We have spent much time Googling around and have found several people with related problems on other operating systems (CentOS and OpenBSD). It appears that OpenSSL has been updated and now requires a longer SSL key to function properly.
This bug on launchpad page may be related.
We have attempted to fix the problem by following the CentOS instructions given here. NOTE: We changed the location of the new dhparams.pem file.
Generate DH parameters file on your server:
openssl dhparam -out /etc/pki/tls/certs/dhparams.pem 1024 Configure sendmail to use this parameters file, and to use only strong ciphers.
Add to /etc/mail/sendmail.mc:
O LOCAL_CONFIG O CipherList=HIGH:!ADH O DHParameters=/etc/pki/tls/certs/dhparams.pem O ServerSSLOptions=+SSL_OP_NO_SSLv2 +SSL_OP_NO_SSLv3 O SSL_OP_CIPHER_SERVER_PREFERENCE O ClientSSLOptions=+SSL_OP_NO_SSLv2 O SSL_OP_NO_SSLv3
Then use make -C /etc/mail/ and service sendmail restart.
This did not appear to improve anything in any way.
Edit:
We turned off TLS by doing the following and sendmail instantly began functioning again. However, this isn't a solution, because we don't want to send plaintext emails.
Add
Try_TLS:1.2.3.4 NO
to /etc/mail/access.
Do a make in /etc/mail and restart sendmail.