4

I'm looking for a way to take gads of inbound SMTP messages and drop them onto an AMQP broker for further routing and processing. The messages won't actually end up in a mailbox, but instead SMTP is used as a message gateway.

I've written a Postfix After-Queue Content Filter in Python that drops the inbound SMTP message onto a RabbitMQ broker. That works well - I get the raw message over a queue and it gets picked up nicely by a consumer. The issue is that the AMQP connection is created and torn down with each message... the Content Filter script gets re-executed from scratch each time. I imagine that will end up being a performance issue.

If I could leverage something re-entrant I could reuse the connection. Or maybe I'm just approaching the whole thing incorrectly...

DeckerEgo
  • 306
  • 1
  • 8

3 Answers3

1

Making an AMQP connection over plain TCP is pretty quick. Perhaps if you're using SSL then it's another story but you sure that enqueueing the raw message onto the AMQP exchange is going to be the bottleneck? My guess would be that actually delivering the message via SMTP is going to be much slower so how fast you can queue things up isn't going to affect the throughput of the system.

If this piece does turn out to be a bottleneck I rather like creating little web servers using Sinatra, or Rack but it sounds like you might prefer a Python based solution. Have the postfix content filter perform a HTTP POST using curl to a webserver, which maintains a persistent connection to the AMQP server.

Of course now you have an extra moving part for which you need to think about monitoring, error handling and security.

Tim Potter
  • 2,437
  • 21
  • 31
  • That's a good point... I wonder what the time difference is between an AMQP construct/teardown vs. an HTTP construct/teardown. No SSL is involved and the exchange (should) already be constructed, so it may not be a huge issue. Just trying to save my successor from bad decisions ;) Don't really care about Python... find with Sinatra for this sorta thing. – DeckerEgo Aug 13 '12 at 13:24
0

Use SwiftMQ. It has a JavaMail bridge which receives your eMails from an IMAP or POP3 account, converts it into a JMS message which then can be consumed by an AMQP 0.9.1 and/or AMQP 1.0 and of course JMS client.

netvault
  • 1
  • I think that would definitely be the way to go if the e-mails ever arrived at a mailbox. However, the goal is for them never to hit a mailbox at all - but instead go straight from SMTP acceptance to the broker. – DeckerEgo Aug 13 '12 at 13:22
0

You can make Postfix deliver all or any of your emails to an external program, where you can throw them anywhere you want. Some examples can be found here.

Community
  • 1
  • 1
sanmai
  • 29,083
  • 12
  • 64
  • 76