0

I have a series of AWS resources that I use to convert pdf files and read tables off them. There is a lambda that reads off an SNS topic message, which has a list of objects, each with a key (filename) and location (s3 address). The lambda then calls a separate package off a private pypi repository, which converts these pdfs to Excel files.

I am using moto to mock the aws resources in the unittest framework. The problem I face is that this external pypi package has pandas and other external dependencies. Also, I seem to get an error saying I can't import my test file.

Here is my test script (test_lambda_function.py):

import unittest
import boto3
import json

from moto import mock_s3, mock_sns
from lambda_function import lambda_handler

SNS_MESSAGE = {
    'location': 's3_address',
    'pdf_files': [
        {
            'bucket': 'input-bucket',
            'key': 'a123456'
        },

        {
            'bucket': 'output-bucket',
            'key': 'b123456'
        }
    ]
}


class TestConverter(unittest.TestCase):
    def setUp(self):
        self.s3 = boto3.client('s3')
        self.sns = boto3.client('sns')

    @mock_s3
    @mock_sns
    def test_lambda_handler(self):
        input_bucket = 'input-bucket'
        output_bucket = 'output-bucket'
        self.s3.create_bucket(Bucket=input_bucket)
        self.s3.put_object(Bucket=input_bucket, Key='file1.pdf', Body=b'These are the contents of file 1')
        self.s3.put_object(Bucket=input_bucket, Key='file2.pdf', Body=b'These are the contents of file 2')

        response = self.sns.create_topic(Name='test-topic')
        topic_arn = response['TopicArn']
        self.sns.publish(TopicArn=topic_arn, Message=json.dumps(SNS_MESSAGE))

        event = {'Records': [{'Sns': {'Message': json.dumps(SNS_MESSAGE)}}]}
        lambda_handler(event, {})

        output_object = self.s3.get_object(Bucket=output-bucket, Key='converted.xlsx')
        self.assertEqual(output_object['Body'].read(),
                         b'File1 converted\nFile2 converted')  

Here is the error message:

ImportError: Failed to import test module: test_lambda_function
Traceback (most recent call last):
  File "C:\PATH\Python\Python310\lib\unittest\loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "C:\PATH\lambda_functions\converter\test_lambda_function.py", line 6, in <module>
    from lambda_function import lambda_handler
  File "C:\PATH\lambda_functions\converter\lambda_function.py", line 5, in <module>
    from capsphere.domain.output.excel import create_worksheet
  File "C:\PATH\lambda_functions\converter\.venv\lib\site-packages\PYPI_PACKAGE\excel.py", line 1, in <module>
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'

Firstly, how do I get the test to load, and secondly how do I then resolve the pandas dependency issue?

clattenburg cake
  • 1,096
  • 3
  • 19
  • 40
  • You should install `pandas` like you install any other dependency. Probably `pip install pandas` in C:\PATH\lambda_functions\converter\.venv – Bert Blommers Apr 04 '23 at 11:40

0 Answers0