How to wait for async actions inside AWS Lambda?

Question

I am trying to process uploaded file in S3. Since getObject is asyncronous main function ends before processing is done, and AWS kills lambda in 3-4 seconds.

Even worse, processing method also has async operations in it - it makes http calls.

On high level, my code looks like:

exports.handler = function(event, context) {
    // Get the object from the event and show its content type
    var bucket = event.Records[0].s3.bucket.name;
    var key = event.Records[0].s3.object.key;
    var params = {
        Bucket: bucket,
        Key: key
    };
    s3.getObject(params, function(err, data) {
        if (err) {
             ...
        } else {
            processFile(data.Body.toString(), 0);
            console.log("ok");
        }
    });
    //need to wait here till processFile is done
};

processFile = function(content, start) {
  ... build url to call
  http.get(url, function(res) {  
    console.log("Got response: " + res.statusCode + ");
    processFile(content, start + 1);
  });
}

I find out that there is async in nodejs but it is not included by amazon; Both require('async') or require('sleep) causes errors.

Lambda timeout configured to 60 seconds, but it exits in 3-4 seconds.

Answer 1

The life of a dev is constantly changing and we now have NodeJS 8 on lambda. For anyone looking at this now check out:

Lambda node 8.10 vs node 6.10 comparison: https://aws.amazon.com/blogs/compute/node-js-8-10-runtime-now-available-in-aws-lambda/

Basics of JS async: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

Even more aws sdk examples: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/using-promises.html

Details on wtf the .promise() method is in the first link: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html#promise-property

Here is my take at a basic example (try pasting into your own lambda):

exports.handler = async (event) => {    
    function wait(){
        return new Promise((resolve, reject) => {
            setTimeout(() => resolve("hello"), 2000)
        });
    }
    
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    
    return 'exiting'
};

The above yields:

As you can see it waited 12 seconds without killing my function :)

TODO more than one thing per await, use Promise.all([]) syntax like this:

exports.handler = async (event) => {
    var uploadPromises = [];
    folder.files.forEach(file => {
        uploadPromises.push( s3.putObject({
            Bucket: "mybucket",
            Key: file.name,
            Body: file.data
        }).promise());
    });

    await Promise.all(uploadPromises);
    return 'exiting'
}; 

Orignal Answer Below

I had the exact same issue on my hands.

The problem is the javascript event loop is empty so Lambda thinks it's done.

This is how I solved this problem. I realize this is not ideal, and I wish there was a better way, but I didn't want to a) add libraries, b) coordinate lambda invocations, or c) switch to another language.

At the end of the day it works.

    exports.handler = (event, context, callback) => {
        var response;
        var callBackCount;

        /*
        Ensures the javascript event loop is never empty.
        This is the key to keeping lambda from exiting early
        */
        setInterval(function(){}, 1000);

        /*
        Tell lambda to stop when I issue the callback.
        This is super important or the lambda funciton will always go until it hits the timeout limit you set.
        */
        context.callbackWaitsForEmptyEventLoop = false;
        
        //My way of determining when I'm done with all calls
        callBackCount = 0;
      
        //My info to return
        response = "";
        
        //Various functions that make rest calls and wait for a response
        asyncFunction1();
        asyncFunction2();
        asyncFunction3();

        //Same for asyncFunction 2 and 3
        function asyncFunction1(){
          response += callBackResponseForThisMethod;
      
          returnResponse();
        }

        function returnReponse(){
            callBackCount++;

            if(callBackCount == 3){
              //Lambda will stop after this as long as    context.callbackWaitsForEmptyEventLoop was set to false 
              callback(null, JSON.stringify(response));
            }
        }

    };

http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html

Answer 2

Using async/await

let AWS = require('aws-sdk');
let lambda = new AWS.Lambda();
let data;
exports.handler = async (event) => {
    try {
        data = await lambda.getAccountSettings().promise();
    }
    catch (err) {
        console.log(err);
        return err;
    }
    return data;
};

Answer 3

async is not included but that does not mean you cannot add it yourself. Simply add the package locally (npm install async), and include the node_modules folder in your ZIP before uploading your Lambda function.

If you want to handle dev dependencies separately (e.g.: test, aws-sdk to execute your function locally, etc), you can add them under devDependencies in your package.json. Also, if you want to automate the process of developing, testing, deploying and promoting your code, these two repos will turn out to be very handy.

Grunt routine to test, package and deploy your lambdas

Command line tool for running and deploying your lambda functions

Answer 4

Think of Lambda simply as a program that you can run in a certain amount of time. The fact that you make asynchronous calls is nice since the (virtual) processor can possibly interleave those calls. However, if any part of your Lambda program takes longer to complete than the time allotted, well, execution will fail. That's the compromise you make and is how Amazon makes money; by selling you more time or memory.

To fix that on your end you can increase the memory your Lambda function is allotted. This not only increases your RAM but also the speed of your virtual processor. Another thing you can do is increase the timeout. AWS Lambda now allows you up to 512 MB of RAM and up to 5 minutes of processing time. As of this post those numbers may have changed so check here for the latest limits. To change this setting, go to your function, then configuration and finally advanced.

Answer 5

I think your lambda function should end with a context.done() call. For instance, try adding it this way:

s3.getObject(params, function(err, data) {
    if (err) {
         ...
        context.done("Error: " + err.stack);
    } else {
        processFile(data.Body.toString(), 0);
        console.log("ok");
        context.done(null, "success");
    }
});

Answer 6

If you want to use require('async'); pack or require('sleep'); pack as well you need to upload your function as a zip file like this:

Creating a Deployment Package (Node.js)

Zip all the content of the folder as I explain in this question as well:

MQTT in AWS Lambda function for Alexa Javascript

About the synchronous processing, you can use require('async'); normally, is just to use the async.series function like this:

    async.series([
    function(callback) {
        // to do the function 1
        callback();
    },
    function(callback) {
        // to do the function 2
        callback();
    },
    function(callback) {
        // to do the function 3
        callback();
    }
], function(err) {
    // to do the function if any error happens...

    if (err) {
        //...
    }
    //....
});

This way the lambda function will works synchronously.

I hope help you.

Answer 7

You might want to make a synchronous call instead; since you seem to be processing your file in the same lambda function.

If for some reason you want to get a callback; you can do that by either directly invoking lambda or via something that would produce a lambda event. Note that lambda functions are supposed be stateless; so you should pass in all of the needed information.