3

I'm playing around with Chalice for the first time as I am trying to evaluate it as a possible replacement framework to migrate my existing Python Flask APIs from EC2 to Lambda.

From an Amazon Linux EC2 instance, I added some dependencies to a virtualenv I'm playing with. I then created a requirements.txt:

botocore==1.7.11
chalice==1.0.2
click==6.6
docutils==0.14
jmespath==0.9.3
MySQL-python==1.2.5
PyMySQL==0.7.11
python-dateutil==2.6.1
six==1.10.0
SQLAlchemy==1.1.14
typing==3.5.3.0

I then tried to deploy with chalice deploy and got:

Creating deployment package.

Could not install dependencies:
MySQL-python==1.2.5
typing==3.5.3.0
You will have to build these yourself and vendor them in
the chalice vendor folder.

Your deployment will continue but may not work correctly
if missing dependencies are not present. For more information:
http://chalice.readthedocs.io/en/latest/topics/packaging.html
........

I then tried to follow linked the docs and for the first problematic dependency MySQL-python==1.2.5 I did the following:

cd vendor/
pip download MySQL-python==1.2.5
pip wheel MySQL-python-1.2.5.zip
rm rm MySQL-python-1.2.5.zip 
unzip MySQL_python-1.2.5-cp27-cp27mu-linux_x86_64.whl 
rm MySQL_python-1.2.5-cp27-cp27mu-linux_x86_64.whl

My vendor folder looks like:

ls vendor
MySQLdb  _mysql_exceptions.py  MySQL_python-1.2.5.dist-info  _mysql.so

and now when I run chalice deploy I get:

(test)[ec2-user@ip-172-31-26-155 test]$ chalice deploy
Creating deployment package.

Could not install dependencies:
MySQL-python==1.2.5
You will have to build these yourself and vendor them in
the chalice vendor folder.

Your deployment will continue but may not work correctly
if missing dependencies are not present. For more information:
http://chalice.readthedocs.io/en/latest/topics/packaging.html

/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: '_mysql.so'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: '_mysql_exceptions.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQL_python-1.2.5.dist-info/RECORD'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQL_python-1.2.5.dist-info/DESCRIPTION.rst'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQL_python-1.2.5.dist-info/WHEEL'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQL_python-1.2.5.dist-info/top_level.txt'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQL_python-1.2.5.dist-info/METADATA'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQL_python-1.2.5.dist-info/metadata.json'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/converters.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/release.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/__init__.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/cursors.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/times.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/connections.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/constants/REFRESH.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/constants/CLIENT.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/constants/ER.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/constants/FIELD_TYPE.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/constants/__init__.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/constants/CR.py'
  zipped.write(full_path, zip_path)
/home/ec2-user/test/test/local/lib/python2.7/site-packages/chalice/deploy/packager.py:110: UserWarning: Duplicate name: 'MySQLdb/constants/FLAG.py'
  zipped.write(full_path, zip_path)

From the documentation, it's not clear to me what I'm doing wrong. Can someone help?

mickzer
  • 5,958
  • 5
  • 34
  • 57
  • That's just an error you can ignore right now. But make sure those packages are available in vendor directory... – Raja Simon Nov 17 '17 at 13:14
  • I am having the same problem, unfortunately, when chalice deploys this zip file, AWS sees the errors and doesn't use it (specifically, I guess the default is for zip to ask whether to replace the file upon extraction) – Cwissy Dec 15 '17 at 08:13
  • Have you tried copying the entire `MySQL-1.2.5...` directory from you `site-packages` virtual environment directory and paste in chalice `vendor` directory? That's what I do and it works for me... – Renato Byrro Jun 19 '18 at 09:55
  • For packages that have code compiled in `C`, you'll need to do this in an instance running Amazon Linux Machine, copy the files and then put in your `vendor`. See [this](https://davidhamann.de/2017/01/27/import-issues-running-python-aws-lambda/) and [this](http://www.perrygeo.com/running-python-with-compiled-code-on-aws-lambda.html). – Renato Byrro Jun 19 '18 at 09:59

2 Answers2

1

Looking at what you have in your directory listing you provided, I noticed you don't have a init.py file. This file identifies the folder as a library file. Put that in your vendors directory.

griff4594
  • 484
  • 3
  • 15
1

You can remove "MySQL-python==1.2.5" from your requirements.txt (since it's already present in your vendor directory)

See this issue in the Chalice repo for more info.

glitchedout
  • 11
  • 4