How to deploy Python AWS Lambda with its dependencies using Javascript AWS CDK?

Question

I have an AWS CDK stack built using javascript. The Stack has multiple lambdas. Most of them are in javascript. I want to add one lambda in python. The python lambda works fine when I don't import external dependencies, but doesn't understand when I install them. I've tried installing the packages in a package folder or a python folder and zipping them as these articles suggested but didn't work:

https://docs.aws.amazon.com/lambda/latest/dg/python-package.html https://aws.amazon.com/premiumsupport/knowledge-center/lambda-import-module-error-python/

The error I'm getting is "Unable to import module 'py1': No module named 'x'" (x refer to any package name I'm trying to import)

My javascript CDK library code is like this:

// javascript lambda
new lambda.Function(this, 'lambda-js1', {
  functionName: `js1`,
  code: lambda.Code.fromAsset('assets/lambdajs'),
  handler: 'js1.handler',
  runtime: lambda.Runtime.NODEJS_14_X,
})

// python lambda
new lambda.Function(this, 'lambda-py1', {
  functionName: `py1`,
  code: lambda.Code.fromAsset('assets/lambdapy'),
  handler: 'py1.handler',
  runtime: lambda.Runtime.PYTHON_3_8,
})

I installed the dependency in assets/lambdapy using

pip install x
pip install --target ./package x
pip install -t python/ x

I zipped them after

My python code is in assets/lambdapy/py1.py

# not sure how to import. none of the below worked
import x
import package.x
import python.x

def handler(event, context):
    return { 
        'statusCode': 200
    }

Upon invocation of the python lambda, I get

{
  "errorMessage": "Unable to import module 'py1': No module named 'x'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}

you need to make the use of Layers in Lambda. Package all your dependencies and the create a layer and upload the package import them in your Lambda https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html — Mohsin Khan, Nov 04 '21 at 00:26

score 4 · Answer 1 · answered Nov 04 '21 at 09:52

4

Use the aws-cdk.aws-lambda-python L2 construct, it installs the dependencies automatically.

Here's the documentation:

https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-python-readme.html

Typescript example form the docs above:

import * as lambda from "@aws-cdk/aws-lambda";
import { PythonFunction } from "@aws-cdk/aws-lambda-python";

new PythonFunction(this, 'MyFunction', {
  entry: '/path/to/my/function', // required
  index: 'my_index.py', // optional, defaults to 'index.py'
  handler: 'my_exported_func', // optional, defaults to 'handler'
  runtime: lambda.Runtime.PYTHON_3_6, // optional, defaults to lambda.Runtime.PYTHON_3_7
});

It will install dependencies from a poetry file, a pipfile, or requiremenets.txt

answered Nov 04 '21 at 09:52

gshpychka

8,523
1
11
31

I didn't try it because it's still in the experimental phase and also requires Docker – anasqadrei Nov 20 '21 at 05:27
3

This is by far the easiest solution. Note that for current releases of CDK (v2.10 at the moment) this is still in alpha, which means you need to install it separately `pip install aws-cdk.aws-lambda-python-alpha`, then import it with `import aws_cdk.aws_lambda_python_alpha as aws_lambda_python`. While this special construct may just be creating a layer itself, I find it messy and added work to need to create a layer in my app just to add deps for a single lambda. – ryanjdillon Dec 10 '21 at 10:30
I'll add that I found myself here because I am creating a lambda with the standard construct `lambda.Function` specifying the code with `Code.fromDockerBuild`, as I need to add some other system libs. Going this route, it appears the only solution may be to add a layer, which is annoying, as the `PythonFunction` construct works so nicely otherwise. – ryanjdillon Dec 10 '21 at 10:35
This should be the accepted answer. Note that if you create your layer separately from the lambda. For example to use in a second lambda, you need to pay attention to `compatible_runtimes` – MikeF Mar 03 '23 at 22:00

score 1 · Accepted Answer · answered Nov 20 '21 at 05:42

My solution

First, I installed the dependency twice (one to work on local machine and one to work on lambda) in assets/lambdapylibs using

# to work on local machine
pip install x

# to work on lambda. it has to be under python folder (assets/lambdapylibs/python)
pip install --target ./python x

Second, I coded assets/lambdapy/py1.py and imported x normally

import x

def handler(event, context):
    return { 
        'statusCode': 200
    }

Third, I added a layer in my javascript CDK library. The layer has the python dependencies

// javascript lambda
new lambda.Function(this, 'lambda-js1', {
  functionName: `js1`,
  code: lambda.Code.fromAsset('assets/lambdajs'),
  handler: 'js1.handler',
  runtime: lambda.Runtime.NODEJS_14_X,
})

// python libs layer
const layerPythonLibs = new lambda.LayerVersion(this, 'layer-python-libs', {
  code: lambda.Code.fromAsset('assets/lambdapylibs'),
  compatibleRuntimes: [lambda.Runtime.PYTHON_3_8],
})

// python lambda
new lambda.Function(this, 'lambda-py1', {
  functionName: `py1`,
  code: lambda.Code.fromAsset('assets/lambdapy'),
  handler: 'py1.handler',
  runtime: lambda.Runtime.PYTHON_3_8,
  layers: [layerPythonLibs],
})

score -1 · Answer 3 · answered Nov 04 '21 at 00:31

-1

Create a Lambda layers for Python. Package all your dependencies in zip file and upload it to the layer created. Please refer the link for the same https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html

answered Nov 04 '21 at 00:31

Mohsin Khan

90
2
11

Thanks for the help. I did something similar – anasqadrei Nov 20 '21 at 05:30

How to deploy Python AWS Lambda with its dependencies using Javascript AWS CDK?

3 Answers3