1

I want to scan file by uising ClamAV from s3 bucket with AWS lambda but size of the local storage in aws lambda that is /tmp 512MB . Im my case i want to download file from s3 bucket in AWS lambda but file size must be grater then 1 GB. Please help me out if anyone knows.

Thanks in Advance

Amin Ali
  • 47
  • 1
  • 8

2 Answers2

3

You could also setup EFS for lambda to have virtual unlimited space which can also be shared with other services, such as EC2, ECS and other Lambda functions.

Marcin
  • 215,873
  • 14
  • 235
  • 294
  • If i'll use EFS, so i can download large file directly to EFS like/efs instead /tmp ? – Amin Ali May 20 '21 at 11:31
  • 1
    @AminAli Yes, that's exactly what you can do. – Marcin May 20 '21 at 11:50
  • 1
    @Marcin AFAIK the container 10 GB is only for the image size. The local storage is still limited to 512 MB – Korgen May 20 '21 at 11:53
  • @Korgen Thanks for clarification. I removed it from the answer. – Marcin May 20 '21 at 12:44
  • Just be aware that EFS performance with a single service accessing it is fairly horrible. It will be a few orders of magnitude slower than accessing a local file system like `/tmp`. If that causes problems (like causing your Lambda function to run longer than 15 minutes) then you'll need to look for other ways to run your application, like ECS+Fargate as the other answer suggests. – Mark B May 20 '21 at 12:48
2

Lambda Limit is 512 Mb Lambda Limits, and cannot be changed.

The following quotas apply to function configuration, deployments, and execution. They cannot be changed.

/tmp directory storage 512 MB

If you need to run task that needs more space, one option would be convert the lambda to a container and run it as a fargate task.

The limits for fargate are 200Gb Fargate Tasks storage information

By default, Amazon ECS tasks hosted on Fargate using platform version 1.4.0 or later receive a minimum of 20 GiB of ephemeral storage. The total amount of ephemeral storage can be increased, up to a maximum of 200 GiB, by specifying the ephemeralStorage parameter in your task definition.

Andreu Gallofré
  • 1,245
  • 9
  • 14