Python multithreaded SMTP proxy

Question

I made a SMTP proxy made of multiple instances of smtpd.SMTPServer running on 5 different ports, receiving, parsing and reinjecting emails in Postfix in a load balanced setup.

All is working well, but I have not been able to run each instance on separated threads. I want to wait about 30/40 seconds between parsing and delivery of the single mail (to further process them), but if i put a time.sleep all instances are blocked.

I would like to switch to parallel processing instead of sequential, in order to have 5 concurrent processing threads, here's the skeleton of my single threaded code:

class My_Proxy(smtpd.SMTPServer):
        def process_message(self, peer, mailfrom, rcpttos, data, decode_data=True):
        ###PARSING AND PROCESSING STUFF...
        time.sleep( 30 ) ###BLOCKING SLEEP
        ###SEND TO FINAL MTA
        server.sendmail(mailfrom, rcpttos, data_rewrite_https)
        server.quit()

#FIVE INSTANCES DIFFERENT PORTS
server1 = My_Proxy(('127.0.0.1', 10027), None)
server2 = My_Proxy(('127.0.0.1', 10029), None)
server3 = My_Proxy(('127.0.0.1', 10031), None)
server4 = My_Proxy(('127.0.0.1', 10033), None)
server5 = My_Proxy(('127.0.0.1', 10035), None)
asyncore.loop()

score 0 · Accepted Answer · answered Jan 30 '20 at 16:25

Solved. For anybody eventually in same trouble: if you want multithreading then you'll need to switch to aiosmtpd based on asyncio and then capable of parallel processing of mails.

Smtpd asyncore loop just keep running sequential, so time.sleep() would sleep entire processing rather than single mail.

The only bad until now is a more complex management of mail content. Where smtpd transparently managed body, encoding, attachments, with aiosmtpd you need to parse separately envelope parts and encoding.

Python multithreaded SMTP proxy

1 Answers1