1

I am using flask, and have a route that sends emails to people. I am using threading to send them faster. When i run it on my local machine it takes about 12 seconds to send 300 emails. But when I run it on lambda thorough API Gateway it times out. here's my code:

import threading

def async_mail(app, msg):
    with app.app_context():
        mail.send(msg)


def mass_mail_sender(order, user, header):
    html = render_template('emails/pickup_mail.html', bruger_order=order.ordre, produkt=order.produkt)
    msg = Message(recipients=[user],
                  sender=('Sender', 'infor@example.com'),
                  html=html,
                  subject=header)
    thread = threading.Thread(target=async_mail, args=[create_app(), msg])
    thread.start()
    return thread

@admin.route('/lager/<url_id>/opdater', methods=['POST'])
def update_stock(url_id):
    start = time.time()
    if current_user.navn != 'Admin':
        abort(403)
    if request.method == 'POST':
        produkt = Produkt.query.filter_by(url_id=url_id)
        nyt_antal = int(request.form['bestilt_hjem'])
        produkt.balance = nyt_antal
        produkt.bestilt_hjem = nyt_antal
        db.session.commit()
        orders = OrdreBog.query.filter(OrdreBog.produkt.has(func.lower(Produkt.url_id == url_id))) \
            .filter(OrdreBog.produkt_status == 'Ikke klar').all()
        threads = []
        for order in orders:
            if order.antal <= nyt_antal:
                nyt_antal -= order.antal
                new_thread = mass_mail_sender(order, order.ordre.bruger.email, f'Din bog {order.produkt.titel} er klar til afhentning')
                threads.append(new_thread)
                order.produkt_status = 'Klar til afhentning'
                db.session.commit()
        for thread in threads:
            try:
                thread.join()
            except Exception:
                pass
        end = time.time()
        print(end - start)
        return 'Emails sendt'
    return ''
bjernie
  • 158
  • 3
  • 13
  • 12 seconds is over the default timeout of 3 seconds for Lambda functions. What are your Lambda settings? – kichik Apr 24 '20 at 22:40

1 Answers1

3

AWS lambda functions designed to run functions within these constraints:

Memory– The amount of memory available to the function during execution. Choose an amount between 128 MB and 3,008 MB in 64-MB increments.

Lambda allocates CPU power linearly in proportion to the amount of memory configured. At 1,792 MB, a function has the equivalent of one full vCPU (one vCPU-second of credits per second).

Timeout – The amount of time that Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds.

To put this in your mail sending multi threaded python code. The function will terminate automatically either when your function execution completes successfully or it reaches configured timeout.

I understand you want single python function to send n number of emails "concurrently". To achieve this with lambda try the "Concurrency" setting and trigger your lambda function through a local script, S3 hosted html/js triggered by cloud watch or EC2 instance.

Concurrency – Reserve concurrency for a function to set the maximum number of simultaneous executions for a function. Provision concurrency to ensure that a function can scale without fluctuations in latency.

https://docs.aws.amazon.com/lambda/latest/dg/configuration-console.html

Important: All above settings will affect your lambda function execution cost significantly. So plan and compare before applying.

If you need any more help, let me know.

Thank you.

HSharma
  • 82
  • 2