18

We are working on a project/framework that aids in deploying and maintaining code in AWS Lambda. I want to build/bundle all node.js code for a lambda function into one js file because:

  • Smaller codebases help with lambda cold start issues
  • Lambda has code zip size limit of 50MB

We don't want to create a custom bundler to do this because there are many options already out there (systemjs,browserify,webpack etc). HOWEVER we are concerned with issues with some node modules not playing well with bundlers/builders.

Specifically aws-sdk has known issues with webpack, says it has browserify support but I've talked to people who have had issues with browserify'ing aws-sdk

We want to pick an existing bundler (or 2), but we want to make sure it works for as many modules/code as possible. We are trying to create a plugin ecosystem for JAWS, so getting this right is important (don't want to turn people off because module X they use wont bundle).

Questions:

  1. How would I go about bundling/building to satisfy these constraints?
  2. Is there any guidelines we can give to consumers of our product to ensure the code they write/use will bundle? Ex: Dynamic require()s cause issues AFAIK.
rynop
  • 50,086
  • 26
  • 101
  • 112
  • Have you tried browserify with the following options: ``` { builtins: false, commondir: false, browserField: false, detectGlobals: false, ignoreMissing: true, } ``` – twilson63 Nov 02 '15 at 18:46

4 Answers4

2

I prefer not to use single file solution, but to upload zip file with all my code packed. I use gulp for this. The code below only uploads production dependencies, excluding development ones and zip archive from previous upload. You may also exclude your test folder or any other file using parameters of gulp.src.

Just to mention, aws-sdk library is in my development dependencies as AWS Lambda already has a copy ;)

var gulp = require('gulp');
var lambda = require('gulp-awslambda');
var zip = require('gulp-zip');
var settings = require('./settings');
var argv = require('yargs').argv;
var p = require('./package.json');

gulp.task('default', function () {
    var profile = argv.profile;
    var src = ['**', '!*.zip', '!node_modules/aws-sdk{,/**}'];
    var i;
    for (i in p.devDependencies) {
        src.push("!node_modules/" + i + "{,/**}");
    }
    if (!profile) {
        profile = settings.aws.defaultProfile;
    }
    return gulp.src(src)
        .pipe(zip('archive.zip'))
        .pipe(lambda(settings.aws.lambda, {
            region: settings.aws.region,
            profile: profile
        }))
        .pipe(gulp.dest('.'));
});
dchertousov
  • 69
  • 5
2

This doesn't directly answer your question, but the serverless project might be exactly what someone in this situation needs.

It allows you to build projects in a normal webpack-style multi-file architecture, then uses a CLI utility to build your project(s) into Lambda-optimized files.

The CLI also handles function initialization, deployment, and a litany of other functions I've not yet even needed. It will even create/configure your triggers (e.g., s3 object creation or setting up a new REST API thru AWS API service).

I only have a few Lambda functions, but even maintaining those was a pain until I started using serverless.

(this is a fawning post, but to be clear I'm not affiliated w/ the project)

Community
  • 1
  • 1
Brandon
  • 7,736
  • 9
  • 47
  • 72
1

aws-sdk-js now officially supports browserify. You can why this is a great thing on my blog.

I have created a serverless plugin called serverless-plugin-browserify that will do all the heavy lifting with very minimal configuration.

To answer the question specifically, I solved the problem with this browserify config:

{
  disable: false, //Not an official option, used as internal option to skip browserify
  exclude: [],    //Not an option, but will use for setting browserify.exclude() if defined in yml
  ignore:  [],    //Not an option, but will use for setting browserify.ignore() if defined in yml

  basedir:          this.serverless.config.servicePath,
  entries:          [],
  standalone:       'lambda',
  browserField:     false,  // Setup for node app (copy logic of --node in bin/args.js)
  builtins:         false,
  commondir:        false,
  ignoreMissing:    true,  // Do not fail on missing optional dependencies
  detectGlobals:    true,  // We don't care if its slower, we want more mods to work
  insertGlobalVars: {      // Handle process https://github.com/substack/node-browserify/issues/1277
    //__filename: insertGlobals.lets.__filename,
    //__dirname: insertGlobals.lets.__dirname,
    process: function() {
    },
  },
  debug:            false,
}

You can see my full code here with a complete example here

rynop
  • 50,086
  • 26
  • 101
  • 112
0

Possible answer might be making the function behave like a bootloader. Make the function download packaged code from S3, unpack it and run it.

Having to touch the code the way you're proposing seems like a bad idea that could result in unpredictable and hard to debug bugs.

adamkonrad
  • 6,794
  • 1
  • 34
  • 41