Problem with Python Zappa deploy to AWS Lambda

Question

I couldn't fix the problem with zappa aws lambda deploy. My zappa settings:

{
    "dev": {
        "aws_region": "us-east-1",
        "django_settings": "photo_crm.settings",
        "profile_name": "default",
        "project_name": "photo-crm",
        "runtime": "python3.10",
        "s3_bucket": "photo-crm-zappa",
        "keep_warm": true,
        "slim_handler": true,
        "events": [
            {
              "function": "photo_crm.tasks.destroy_galleries_by_terms",
              "expression": "cron(0 0 * * ? *)"
            },
            {
              "function": "photo_crm.tasks.birthdays_telegram_bot",
              "expression": "cron(0 8 * * ? *)"
              }
        ],
        "remote_env": "s3://bucket_link",
        "exclude": [".env"]
    }
}

and I've got the error in the end:

Error: Warning! Status check on the deployed lambda failed. A GET request to '/' yielded a 500 response code.

Part of Logs

Calling tail for stage dev..
[1685898767317] INIT_START Runtime Version: python:3.10.v5 Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:51b59a64d1fa5125d324f5fe77fbc805ea96f487f1d112fe4bf3f60323b552cb
[1685898767841] Instancing..
[1685898772523] [ERROR] ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2._psycopg'
Traceback (most recent call last):
  File "/var/task/handler.py", line 657, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 241, in lambda_handler
    handler = global_handler or cls()
  File "/var/task/handler.py", line 150, in __init__
    wsgi_app_function = get_django_wsgi(self.settings.DJANGO_SETTINGS)
  File "/var/task/zappa/ext/django_zappa.py", line 13, in get_django_wsgi
    return get_wsgi_application()
  File "/var/task/django/core/wsgi.py", line 12, in get_wsgi_application
    django.setup(set_prefix=False)
  File "/var/task/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/var/task/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/var/task/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
  File "/var/lang/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/var/task/django/contrib/auth/models.py", line 3, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/var/task/django/contrib/auth/base_user.py", line 49, in <module>
    class AbstractBaseUser(models.Model):
  File "/var/task/django/db/models/base.py", line 141, in __new__
    new_class.add_to_class("_meta", Options(meta, app_label))
  File "/var/task/django/db/models/base.py", line 369, in add_to_class
    value.contribute_to_class(cls, name)
  File "/var/task/django/db/models/options.py", line 231, in contribute_to_class
    self.db_table, connection.ops.max_name_length()
  File "/var/task/django/utils/connection.py", line 15, in __getattr__
    return getattr(self._connections[self._alias], item)
  File "/var/task/django/utils/connection.py", line 62, in __getitem__
    conn = self.create_connection(alias)
  File "/var/task/django/db/utils.py", line 193, in create_connection
    backend = load_backend(db["ENGINE"])
  File "/var/task/django/db/utils.py", line 113, in load_backend
    return import_module("%s.base" % backend_name)
  File "/var/lang/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/var/task/django/db/backends/postgresql/base.py", line 28, in <module>
    raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
[1685898787480] [DEBUG] 2023-06-04T17:13:07.479Z c44e96dd-13ca-4d3b-9bff-563e2243605f Zappa Event: {'time': '2023-06-04T17:12:27Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': '652e4998-2c71-8fb1-7983-965bf919ff83', 'kwargs': {}}
[1685898787480] [DEBUG] 2023-06-04T17:13:07.480Z c44e96dd-13ca-4d3b-9bff-563e2243605f Zappa Event: {}
[1685899027564] [DEBUG] 2023-06-04T17:17:07.564Z 16dadc58-15b9-43dd-b895-d0b0f0079ee3 Zappa Event: {'time': '2023-06-04T17:16:27Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': 'af7af3e3-1d6a-eec9-3d14-0f42fcfa89f0', 'kwargs': {}}
[1685899027564] [DEBUG] 2023-06-04T17:17:07.564Z 16dadc58-15b9-43dd-b895-d0b0f0079ee3 Zappa Event: {}
[1685899267381] [DEBUG] 2023-06-04T17:21:07.380Z 894649a6-0713-4832-8329-68a10f226023 Zappa Event: {'time': '2023-06-04T17:20:27Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': '37bd8c4a-b7fb-765a-3b28-d82f958836fd', 'kwargs': {}}
[1685899267381] [DEBUG] 2023-06-04T17:21:07.381Z 894649a6-0713-4832-8329-68a10f226023 Zappa Event: {}
[1685899507348] [DEBUG] 2023-06-04T17:25:07.348Z 8b05ba65-ef08-40f9-955f-eaad6fcbae3d Zappa Event: {'time': '2023-06-04T17:24:27Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': '775887d3-2e0f-1b1c-02a1-34253228b812', 'kwargs': {}}
[1685899507348] [DEBUG] 2023-06-04T17:25:07.348Z 8b05ba65-ef08-40f9-955f-eaad6fcbae3d Zappa Event: {}
[1685899747589] [DEBUG] 2023-06-04T17:29:07.588Z df7214b1-7c94-4b38-86f1-72f24f321540 Zappa Event: {'time': '2023-06-04T17:28:27Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': 'bc906246-d27c-cf79-38f0-876f137aea30', 'kwargs': {}}
[1685899747589] [DEBUG] 2023-06-04T17:29:07.588Z df7214b1-7c94-4b38-86f1-72f24f321540 Zappa Event: {}
[1685899987314] [DEBUG] 2023-06-04T17:33:07.314Z 417be60c-1886-4c9e-9a06-e4b9bf79f983 Zappa Event: {'time': '2023-06-04T17:32:27Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': 'ca102be5-f802-2ac3-19ed-bb9e3e73f247', 'kwargs': {}}
[1685899987314] [DEBUG] 2023-06-04T17:33:07.314Z 417be60c-1886-4c9e-9a06-e4b9bf79f983 Zappa Event: {}
[1685900227362] [DEBUG] 2023-06-04T17:37:07.362Z fb63ecd4-acfc-4ac1-ae82-43be01f63809 Zappa Event: {'time': '2023-06-04T17:36:27Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': '9b2d7f20-9410-fd17-720c-4ad6fc63716c', 'kwargs': {}}
[1685900227362] [DEBUG] 2023-06-04T17:37:07.362Z fb63ecd4-acfc-4ac1-ae82-43be01f63809 Zappa Event: {}
[1685900517411] INIT_START Runtime Version: python:3.10.v5 Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:51b59a64d1fa5125d324f5fe77fbc805ea96f487f1d112fe4bf3f60323b552cb
[1685900517937] Instancing..
[1685900522594] [ERROR] ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2._psycopg'
Traceback (most recent call last):
  File "/var/task/handler.py", line 657, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 241, in lambda_handler
    handler = global_handler or cls()
  File "/var/task/handler.py", line 150, in __init__
    wsgi_app_function = get_django_wsgi(self.settings.DJANGO_SETTINGS)
  File "/var/task/zappa/ext/django_zappa.py", line 13, in get_django_wsgi
    return get_wsgi_application()
  File "/var/task/django/core/wsgi.py", line 12, in get_wsgi_application
    django.setup(set_prefix=False)
  File "/var/task/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/var/task/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/var/task/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
  File "/var/lang/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/var/task/django/contrib/auth/models.py", line 3, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/var/task/django/contrib/auth/base_user.py", line 49, in <module>
    class AbstractBaseUser(models.Model):
  File "/var/task/django/db/models/base.py", line 141, in __new__
    new_class.add_to_class("_meta", Options(meta, app_label))
  File "/var/task/django/db/models/base.py", line 369, in add_to_class
    value.contribute_to_class(cls, name)
  File "/var/task/django/db/models/options.py", line 231, in contribute_to_class
    self.db_table, connection.ops.max_name_length()
  File "/var/task/django/utils/connection.py", line 15, in __getattr__
    return getattr(self._connections[self._alias], item)
  File "/var/task/django/utils/connection.py", line 62, in __getitem__
    conn = self.create_connection(alias)
  File "/var/task/django/db/utils.py", line 193, in create_connection
    backend = load_backend(db["ENGINE"])
  File "/var/task/django/db/utils.py", line 113, in load_backend
    return import_module("%s.base" % backend_name)
  File "/var/lang/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/var/task/django/db/backends/postgresql/base.py", line 28, in <module>
    raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
[1685900532098] [DEBUG] 2023-06-04T17:42:12.098Z 43e4e7c6-6b5f-4498-b693-849f3ea2ad93 Zappa Event: {'time': '2023-06-04T17:41:52Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': '08490cf9-127a-5479-2b26-aeed47d1dc41', 'kwargs': {}}
[1685900532098] [DEBUG] 2023-06-04T17:42:12.098Z 43e4e7c6-6b5f-4498-b693-849f3ea2ad93 Zappa Event: {}
[1685900772224] [DEBUG] 2023-06-04T17:46:12.224Z 31959fd0-cdb4-46bc-bda8-df6caca90923 Zappa Event: {'time': '2023-06-04T17:45:52Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': 'bcb10b79-8151-092e-b5f2-6999cb804414', 'kwargs': {}}
[1685900772225] [DEBUG] 2023-06-04T17:46:12.225Z 31959fd0-cdb4-46bc-bda8-df6caca90923 Zappa Event: {}
[1685900952216] INIT_START Runtime Version: python:3.10.v5 Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:51b59a64d1fa5125d324f5fe77fbc805ea96f487f1d112fe4bf3f60323b552cb
[1685900952813] Instancing..
[1685900957336] [ERROR] ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2._psycopg'
Traceback (most recent call last):
  File "/var/task/handler.py", line 657, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 241, in lambda_handler
    handler = global_handler or cls()
  File "/var/task/handler.py", line 150, in __init__

[1685959933039] [DEBUG] 2023-06-05T10:12:13.039Z ed8a02c9-b8fa-46e1-9b3b-6c624aa040f7 Zappa Event: {'time': '2023-06-05T10:11:39Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': 'c02f90e9-51a8-6b45-def1-ed21c762e16b', 'kwargs': {}}
[1685959933039] [DEBUG] 2023-06-05T10:12:13.039Z ed8a02c9-b8fa-46e1-9b3b-6c624aa040f7 Zappa Event: {}
[1685960177318] [DEBUG] 2023-06-05T10:16:17.318Z 9dd6ca45-fd8c-4e03-b48d-d4b51a2b1c16 Zappa Event: {'time': '2023-06-05T10:15:39Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': '0af73aef-79df-ea41-1bab-cdd4a2e340f5', 'kwargs': {}}
[1685960177318] [DEBUG] 2023-06-05T10:16:17.318Z 9dd6ca45-fd8c-4e03-b48d-d4b51a2b1c16 Zappa Event: {}
[1685960412969] [DEBUG] 2023-06-05T10:20:12.969Z 00001b42-2e4d-4d31-9f8c-c3c73d98275b Zappa Event: {'time': '2023-06-05T10:19:39Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': 'c0890769-5891-4af1-afbd-70c7924644d4', 'kwargs': {}}
[1685960412969] [DEBUG] 2023-06-05T10:20:12.969Z 00001b42-2e4d-4d31-9f8c-c3c73d98275b Zappa Event: {}
[1685960652875] [DEBUG] 2023-06-05T10:24:12.875Z 571376f4-f9d5-478e-b78b-129ccc47e54a Zappa Event: {'time': '2023-06-05T10:23:39Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': 'd0f64a10-d2c9-730e-5c29-441e0b1b23a8', 'kwargs': {}}
[1685960652876] [DEBUG] 2023-06-05T10:24:12.875Z 571376f4-f9d5-478e-b78b-129ccc47e54a Zappa Event: {}
[1685960892828] [DEBUG] 2023-06-05T10:28:12.828Z 27b5a509-0257-49c8-b658-00e4e9359aaf Zappa Event: {'time': '2023-06-05T10:27:39Z', 'detail-type': 'Scheduled Event', 'source': 'aws.events', 'ac
count': '058374175047', 'region': 'us-east-1', 'detail': {}, 'version': '0', 'resources': ['arn:aws:events:us-east-1:058374175047:rule/photo-crm-dev-zappa-keep-warm-handler.keep_warm_callback']
, 'id': 'a000893c-8ed5-fb17-3ffc-0059715de2dd', 'kwargs': {}}
[1685960892828] [DEBUG] 2023-06-05T10:28:12.828Z 27b5a509-0257-49c8-b658-00e4e9359aaf Zappa Event:

How can I fix it? And after first deploy zapa zip was like 30Mb, every time its gets bigger and bigger. Now its 72Mb, but files still the same! I've got the message Warning: Application zip package is likely to be too large for AWS Lambda. Try setting "slim_handler" to true in your Zappa settings file. requirements.txt

amqp==5.1.1
APScheduler==3.10.0
argcomplete==3.0.8
asgiref==3.6.0
async-timeout==4.0.2
aws-psycopg2==1.3.8
billiard==3.6.4.0
boto3==1.26.138
botocore==1.29.138
certifi==2022.12.7
cfn-flip==1.3.0
charset-normalizer==3.1.0
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.6
Django==4.1
django-appconf==1.0.5
django-ckeditor==6.4.2
django-cleanup==6.0.0
django-cors-headers==3.14.0
django-decorator-include==3.0
django-extensions==3.2.1
django-imagekit==4.1.0
django-js-asset==2.0.0
django-resized==1.0.2
django-s3-storage==0.14.0
django-storages==1.13.2
djangorestframework==3.14.0
durationpy==0.5
flake8==6.0.0
hjson==3.1.0
idna==3.4
jmespath==1.0.1
kappa==0.6.0
kombu==5.2.4
MarkupSafe==2.1.2
mccabe==0.7.0
pilkit==2.0
Pillow==9.3.0
placebo==0.9.0
prompt-toolkit==3.0.38
psycopg2==2.9.3
psycopg2-binary==2.9.6
pycodestyle==2.10.0
pyflakes==3.0.1
pyTelegramBotAPI==4.10.0
python-dateutil==2.8.2
python-decouple==3.8
python-slugify==8.0.1
pytz==2022.7.1
pytz-deprecation-shim==0.1.0.post0
PyYAML==6.0
pyzmq==25.1.0
requests==2.28.2
s3transfer==0.6.1
sentry-sdk==1.20.0
six==1.16.0
sqlparse==0.4.3
text-unidecode==1.3
toml==0.10.2
tqdm==4.65.0
transliterate==1.10.2
troposphere==4.3.2
tzdata==2022.7
tzlocal==4.2
urllib3==1.26.15
vine==5.0.0
wcwidth==0.2.6
Werkzeug==2.3.4
zappa==0.57.0

If the deployed ZIP file is getting larger, it seems highly unlikely that the constituent files are the same (in both filename and size). — jarmod, Jun 05 '23 at 16:08
Also see https://stackoverflow.com/questions/44855531/no-module-named-psycopg2-psycopg-modulenotfounderror-in-aws-lambda — jarmod, Jun 05 '23 at 16:10
Thanks, it was helpful. I delete psycopg2 from my requirements, and error gone. Also there were some problems with remote_env. — Anna Berezko, Jun 08 '23 at 06:55

Problem with Python Zappa deploy to AWS Lambda

0 Answers0