Sharp on aws lambda. No space left on device while using .toBuffer

Question

I'm trying to resize images with a resolution of 16384x16384 and higher using nodejs sharp. While the function runs fine on a local machine, the problem appears on aws lambda.

I get this error "Runtime.UnhandledPromiseRejection: Error: wbuffer_write: write failed",
"unix error: No space left on device"

The machine has 2024mb memory and works fine on lower resolution images but throws the error using bigger resolutions. The output resolutions are supposed to be 8192 and 4096

I've tried changing the sharp.cache memory, rewriting the function so it sends the file right after .toBuffer yet it didn't help since it throws the error during the .toBuffer function.

I'm a bit out of ideas and would greatly appreciate help.

The problematic bit of code:

async function changeResolution(){
        try { 
            var key = './image.jpg';
            var image = sharp(key, { limitInputPixels: false });
            return await image
            .resize(8192,8192)
            .toBuffer()
            .then(console.log("I do not get here"))
        } catch (error) {
            console.log(error);
            return;
        }
    }

How big are the both the source files and the destination files? This sounds like you're running out of disk space on the Lambda. A totally uncompressed 16384x16384 JPG would be over a 1GB and you are limited to 512MB on a Lambda. — stdunbar, Dec 11 '20 at 23:24

score 0 · Answer 1 · answered Dec 11 '20 at 23:25

Memory is not disk space.

No space left on device

This error indicates that the Lambda has no disk space left. Typically, if you have to write files to a disk in a Lambda, you write it to /tmp. There you have 512MB of storage and it should be the only place where you can actually write files to.

Looking at your code, you are trying to write your image outside of /tmp and I guess there is not enough or no space left for you to write to.

First, you should try to change your code to store your image at /tmp/image.jpg.

If the resulting image is larger than 512MB, you are running out of options. You could mount an EFS volume to your Lambda.

The example code *reads* an image from "./image.jpg" and resizes to a buffer. The reason I found this SO is that I get the same error, and I load the image from a buffer. — dsvensson, Feb 10 '22 at 15:08

Sharp on aws lambda. No space left on device while using .toBuffer

1 Answers1