AWS SDK Crashing Node app if item doesn't exist in s3

Question

I'm trying to use an S3 to store my application images and files. The thing is that even though I am using try...catch, if I try to fetch a file that doesn't exist, it crashes because access was denied. This is the function I use to get the files:

export const getImage = (req: Request, res: Response, next: NextFunction) => {
    try{
        const key = req.query.key;

        if(!key) throw new HttpException(403, 'Hubo un error al cargar la imagen');
        
        const readStream = getFileStream(key);
        if(readStream) {
            readStream.pipe(res);
        } else {
            throw new HttpException(403, 'Hubo un error al cargar la imagen');
        }
    }catch(err){
        next(err);
    }
}

And the getFileStream function is:

export const getFileStream = (fileKey: string) => {
    const downloadParams = {
        Key: fileKey,
        Bucket: bucketName
    }

    return s3.getObject(downloadParams).createReadStream();
}

The error is:

\node_modules\aws-sdk\lib\services\s3.js:718
      resp.error = AWS.util.error(new Error(), {
                                  ^
AccessDenied: Access Denied

What gets node to say:

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! typescriptapi@1.0.0 start: `ts-node src/app.ts`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the typescriptapi@1.0.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

I tried adding try...catch in the getFileStream function but that didn't work either.

Nir Alfasi suggestion:

turned get Image into an async function and tried doing this:

try{
    const readStream = await getFileStream(key);
    if(readStream) {
        readStream.pipe(res);
    }
} catch(err) { console.log(err)}

Also tried not adding a second try...catch and just using the one that is already there but didn't work either. Same error, Access Denied and crashes.

D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\services\s3.js:718
      resp.error = AWS.util.error(new Error(), {
                                  ^
AccessDenied: Access Denied
    at Request.extractError (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\services\s3.js:718:35)
    at Request.callListeners (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
    at Request.emit (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
    at Request.emit (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\request.js:688:14)
    at Request.transition (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\request.js:22:10)
    at AcceptorStateMachine.runTo (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\state_machine.js:14:12)
    at D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\state_machine.js:26:10
    at Request.<anonymous> (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\request.js:38:9)
    at Request.<anonymous> (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\request.js:690:12)
    at Request.callListeners (D:\Personal\Yo\Paideia\ed\TypeScript\API\node_modules\aws-sdk\lib\sequential_executor.js:116:18)
npm ERR! errno 1
npm ERR! typescriptapi@1.0.0 start: `ts-node src/app.ts`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the typescriptapi@1.0.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

score 2 · Answer 1 · answered May 25 '21 at 14:27

2

createReadStream() returns a promise, which means that we can await on it and also surround it with try/catch to handle errors.

Try changing:

const readStream = getFileStream(key);

to:

try {
    const readStream = await getFileStream(key);
} catch (e) {
    if (e.statusCode === 404) {
        // handle item doesn't exist
        // ...
    }
}

answered May 25 '21 at 14:27

Nir Alfasi

53,191
11
86
129

1

Note, that for using `await` you have to make the surrounding function `async` (which currently isn't the case) – derpirscher May 25 '21 at 14:29
Okay, thank you. Currently using try catch in it but will try to await it. – Tomas Barreiro May 25 '21 at 14:34
No, it's not working either. Going to edit my question with the code on how I tried – Tomas Barreiro May 25 '21 at 14:36
1

@TomasBarreiro "didn't work" is very general and doesn't provide any information into what exactly didn't work. Please update the question with the error that console printed and any other information you suspect can be useful! – Nir Alfasi May 25 '21 at 15:34
@NirAlfasi I edited the question, the Edit2 is your suggestion. – Tomas Barreiro May 25 '21 at 15:35
1

@TomasBarreiro please read my last comment again :) – Nir Alfasi May 25 '21 at 15:36
@NirAlfasi Oh, sorry. I didn't paste it again because it's the same output than before. That's why I said: 'Same error, access denied and crashes.' Going to edit it and add it anyway it – Tomas Barreiro May 25 '21 at 15:46
@TomasBarreiro please post the entire error, including stacktrace if available, BTW, are you sure that this error raises only when the file doesn't exist? I suspect that "NoSuchKey" error should be returned not "Access denied". An Access Denied error is usually raised when there are permission issues (regardless of the existence of the file) – Nir Alfasi May 25 '21 at 15:48
@NirAlfasi Updated. Yes, If i load the page I get the images correctly. If I edit the document and change the url to another key that doesn't exist, it crashes. – Tomas Barreiro May 25 '21 at 15:56
1

@TomasBarreiro this could very much be an IAM role issue, more details in: https://stackoverflow.com/questions/61681230/serverless-s3-upload-access-denied – Nir Alfasi May 25 '21 at 16:02
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/232882/discussion-between-tomas-barreiro-and-nir-alfasi). – Tomas Barreiro May 25 '21 at 19:06

score 1 · Accepted Answer · answered May 27 '21 at 10:13

Nir Alfasi helped me and I found a working answer. The 403 error gets thrown because I didn't have permissions to access a list of existing bucket items, so, if I granted my self that permission I would get a 404. Now, to handle the error I had to add an event listener:

const readStream = getFileStream(key);
if(readStream) {
    readStream.createReadStream().on('error', e => {
        console.log(e);
    }).pipe(res);
}

And changed getFileStream to only get the object:

export const getFileStream = (fileKey: string) => {
    const downloadParams = {
        Key: fileKey,
        Bucket: bucketName
    }

    return s3.getObject(downloadParams);
}

Answer found here

AWS SDK Crashing Node app if item doesn't exist in s3

2 Answers2