0

I have a web application based on flask together with flask-mail (https://pythonhosted.org/flask-mail/) to send emails. I am running a cronjob every 5min which checks for updates in the database and if it finds updates it sends a bunch of emails to all users which are affected by these updates. It can happen that the number of emails send is so large that the job does not finish within 5min, which means that cron is starting another job in parallel. My understanding is that cron does not kill the old job if it is still running after 5min. However, what seems to happen is that the connection for sending emails gets closed in the old job? Meaning that in that case not all users get the email. Here is how I send the emails

users = models.User.query.filter_by(query_email_notification=1).all()
if users:
    # Bulk emails... keep connection open
    with app.app_context():
        with mail.connect() as conn:
            for user in users:
                subject = "subject"
                message = 'Hi'
                msg = Message(recipients=[user.email],
                      body=message,subject=subject, sender='me@gmail.com')
                conn.send(msg) 

which is basically following the main example on the flask-mail website. However, whenever the new cronjob starts the connection closes and no more emails are send. Does anybody understand why this happens and how I can prevent it? One solution would be to make sure that cron only starts a new job if the old one has finished. But that seems messy with cron. Is there some other software which can do that? thanks carl

carl
  • 4,216
  • 9
  • 55
  • 103
  • 2
    How do you know "the connection closes"? Is an error message produced? If so, please copy-paste the exact error message into your question. – Robᵩ Mar 01 '16 at 16:25

1 Answers1

1

One can use flock to serialize long-running cron jobs. See, for example, https://serverfault.com/a/461637/82655

Community
  • 1
  • 1
Robᵩ
  • 163,533
  • 20
  • 239
  • 308