3

We have a Lambda function in python using below libraries

pyOpenSSL==18.0.0

protobuf==3.14.0

The python code is less than 100KB, but with the above dependencies included the size goes to more than 4MB.

Requirement is to deploy this Lambda on a CloudFront distribution (Lambda@Edge) for which size of function with dependencies cannot exceed 1MB.

Is there any way to achieve this?

Any help is appreciated. Any help is appreciated.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Vivek
  • 175
  • 1
  • 9
  • There are several ways to reduce the size. For one: don't use those libraries if you can use standard library to achieve the same. Or compress stuff, strip stuff. But without more context, it is hard to tell which one would be best for you. For starters, you might want to explain for what exactly you need those libraries? For Lambdas@Edge I found that you should try to rely on your runtimes standard library as much as possible. – Jens Jan 28 '21 at 10:02
  • Both these dependencies are needed for this lambda. I was looking at any compression technique to reduce size below 1MB. Without compression the size is around 11MB, with using zip command the zip size comes to around 4MB. – Vivek Jan 29 '21 at 10:45

1 Answers1

0

You'll probably have to modify your deployment tooling a lot.

If these libraries have any compiles binaries, you can remove the debug symbols from them with the strip command (which modifies the files in place)

Something like:

find ./ -name '*.so' -type -f -exec strip "{}" \;

You can also try the same by for *.so*, since there are other files that end in .so.400. Although I have seen cases where this broke the files. So ensure you test well.

Have a look at the size of those libraries. (e.g. pip install pyOpenSSL --target test). Look at what's inside. Look at which folders are the biggest. GUI apps like this make that easy. If you're on a terminal, you may want something like du -s ./ | sort -n.

If there are folders and files that are only for unit testing, try deleting them. (e.g. any folders called test or tests.) Similarly, you can delete README.* files, or other documentation.

I don't know about Lambda@Edge, but for normal lambda, the AWS SDKs are already installed. So if your dependencies depend on the AWS SDKs, you're doubling up, and can delete them. I doubt these ones do though.

An extreme workaround is to zip up your dependencies separately to your main code. Then when the lambda runs, the first thing you do is download the zip from S3, unzip it, and then import those libraries. If Lambda@Edge is like normal lambda, that download will be cached within each Lambda VM. So for a frequently used Lambda, that won't cost anything on warm starts.

falsePockets
  • 3,826
  • 4
  • 18
  • 37