AWS SAM local invoke doesn't work when python venv has dependencies

Question

I am trying to run a very simple python lambda function locally to learn how AWS SAM local invoke works and test my code.

I have a very simple file structure that more or less looks like this:

|- env/
|- lambda_function.py
|- pyproject.toml
|- setup.cfg
|- template.yaml
|- test_data.json

My lambda_function.py looks like this:

import pandas as pd


def lambda_handler(event, context):
    print(event)
    print(pd.DataFrame({'c1': [1, 2, 3]}))
    return {}

My template.yaml looks like the following:

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  MyFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: lambda_function.lambda_handler
      Runtime: python3.8

And my setup.cfg defines the dependencies of my lambda, which at the moment is just pandas. I have pandas installed in my virtual env and it lives inside env/lib/site-packages/...

However, whenever I run the following:

sam local invoke -e ./test_data.json

I get the following error:

Invoking lambda_function.lambda_handler (python3.8)
Skip pulling image and use local one: amazon/aws-sam-cli-emulation-image-python3.8:rapid-1.27.2.

Mounting /Users/XXXXXXXX/XXXXXX/XXXXXXX/[current folder] as /var/task:ro,delegated inside runtime container
START RequestId: 10680278-8001-4a51-87a5-d5fa8a2ab2c6 Version: $LATEST
Traceback (most recent call last): Unable to import module 'lambda_function': No module named 'pandas'
END RequestId: 10680278-8001-4a51-87a5-d5fa8a2ab2c6
REPORT RequestId: 10680278-8001-4a51-87a5-d5fa8a2ab2c6  Init Duration: 0.18 ms  Duration: 130.39 ms     Billed Duration: 200 ms Memory Size: 128 MB     Max Memory Used: 128 MB     
{"errorMessage": "Unable to import module 'lambda_function': No module named 'pandas'", "errorType": "Runtime.ImportModuleError", "stackTrace": []}%

(I have X'd out some personal info) How do I get the SAM CLI to recognise the dependencies in the virtual env?

score 0 · Answer 1 · answered Aug 22 '21 at 07:05

You have to add a layer to the Lambda function for it to work with pandas and other libraries.

Download the last awswrangler-layer zip file from the aws-github-repo.
In AWS-Lambda-Layers create a layer and upload the zip file.
Add the layer to the lambda function.

Here is a video of the process.

AWS SAM local invoke doesn't work when python venv has dependencies

1 Answers1