1

Here is the problem I am facing with. We are having a postfix server that needs to parse emails forwarded from some user's account and extract some data from it. Usually there are around 200 emails per user. We have tested it with 5 users and this all was good, but what to do if the number of user reaches some greater number, for example 10000 or 100000? Do you have any ideas how to make the postfix solution scalable so it could support this heavy load.

Our current postfix server resource is Ubuntu 10.04 machine with 512MB of RAM.

Best regards, Mladjo

Mladen
  • 25,578
  • 11
  • 39
  • 48

1 Answers1

5

Postfix is a mailer. Not a data miner, arbitrary string parser or general purpose light bulb. When receiving 10000 letters, you - the mentally unstable postal worker - do not want to open the letters, read them, cut out some parts, close them and then deliver them. You want to figure out if they're yours to deliver and put them in the right pile. For the other task, you call on your buddy Cron, who's dating Ms. Perl and have all the right features for previously mentioned tasks.

Mel
  • 6,077
  • 1
  • 15
  • 12
  • :) Nice one. Postfix is only filtering emails and matching if source and destination are legit, then I forward the email to procmail which sends the email contents to php parser for data extraction. – Mladen May 09 '11 at 12:53
  • Then you need to make sure that the hand off to the parser does not hold the delivery by calling the parser in the background. Still, I would involve cron and do batch parsing jobs, simply because each mail delivery now starts a parser, which may hang around longer then the delivery, so per incoming mail you have local(8) from postfix and one or more parsers running. This quickly will consume your 512MB. – Mel May 10 '11 at 01:37
  • 1
    To clarify, I've done something similar: process failure mails that come in from a subscriber mail list and updated the database with a fail count. I switched to batched processing after day 1 for exactly this reason. – Mel May 10 '11 at 01:42