0

I am using lambda edge to handle image compression with Sharp. The code works for now, but when I tried to add a new function to parse query parameter to let the user define the quality of the compression, Lambda/Cloudfront starts giving me a notice that the key is not exist even though it does exist.

The path that was used as an example is:

/compress/480/uploads/1000491600869812260.jpg?quality=30

Error that shows up on the browser:

<Error>
  <Code>NoSuchKey</Code>
  <Message>The specified key does not exist.</Message>
  <Key>compress/480/uploads/1000491600869812260.jpg</Key>
  <RequestId>5KPMBD6RNETZCA3Z</RequestId>
  <HostId>
    brMd/eCi6uv9s3VIl4IRHg7FlIytNA8DkgPjGfGrej4SkUsMxuEm1YHGEEll5rydO24gecIOTtE=
  </HostId>
</Error>

Errors log from cloudfront:

#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version fle-status fle-encrypted-fields c-port time-to-first-byte x-edge-detailed-result-type sc-content-type sc-content-len sc-range-start sc-range-end
2021-06-09  06:06:43    ORD52-C3    689 182.253.36.23   GET d32xc09eirob59.cloudfront.net   /compress/480/uploads/1000491600869812260.jpg   404 -   Mozilla/5.0%20(Macintosh;%20Intel%20Mac%20OS%20X%2010_15_7)%20AppleWebKit/605.1.15%20(KHTML,%20like%20Gecko)%20Version/14.1.1%20Safari/605.1.15 quality=10  -   Error   FPFQE5Z-XuBeAK61KaJbNqDAbypyo3BhrH7xom7GZik--UgESIVQFw==    d32xc09eirob59.cloudfront.net   http    426 3.726   -   -   -   Error   HTTP/1.1    -   -   54708   3.726   Error   application/xml -   -   -

In the code below, if I comment the lines that call the function to parse the quality from query parameter (marked as "The problematic line" in the code), the code works again. But, from my point of view, there is nothing wrong with the code since it is a simple regex to fetch a value.

Is there any limitation or constraint in the AWS lambda that makes it behave like that? Is there anything that I can do to make it work?

P.S. I already tried to use URL and querystring library to parse the path, but it always shows me LambdaException error, hence why I try to parse it manually with regex

Problematic line/function:

const getQuality = (path) => {
  const match = path.match(/quality=(\d+)/)
  const quality = parseInt(match[1], 10)
  return quality
}

const quality = getQuality(path)

Full code:

'use strict'

const AWS = require('aws-sdk')
const S3 = new AWS.S3({ signatureVersion: 'v4' })
const Sharp = require('sharp')

const BUCKET = 'some-bucket'
const QUALITY = 70

// Image types that can be handled by Sharp
const SUPPORTED_IMAGE_TYPES = ['jpg', 'jpeg', 'png', 'gif', 'webp', 'svg', 'tiff']
const JSON_CONTENT_HEADER = [{ key: 'Content-Type', value: 'application/json' }]
const WEBP_CONTENT_HEADER = [{ key: 'Content-Type', value: 'image/webp' }]

const getOriginalKey = (path) => {
  const match = path.match(/\/(\d+)\/([A-Za-z0-9_\-]+)\/([A-Za-z0-9_\-]+)\.(\w+)\??/)

  const imageWidth = parseInt(match[1], 10)
  const prefix = match[2]
  const imageName = match[3]
  const imageFormat = match[4]

  const originalKey = `${prefix}/${imageName}.${imageFormat}`
  return { originalKey, imageWidth, imageFormat }
}

const getQuality = (path) => {
  const match = path.match(/quality=(\d+)/)
  const quality = parseInt(match[1], 10)
  return quality
}

const responseUpdate = (
  response,
  status,
  statusDescription,
  body,
  contentHeader,
  bodyEncoding = undefined
) => {
  response.status = status
  response.statusDescription = statusDescription
  response.body = body
  response.headers['content-type'] = contentHeader
  if (bodyEncoding) {
    response.bodyEncoding = bodyEncoding
  }

  return response
}

exports.handler = async (event, context, callback) => {
  let { request, response } = event.Records[0].cf
  const { uri } = request
  const headers = response.headers

  console.log(JSON.stringify({ status_code: response.status, uri }))

  // NOTE: Check whether the image is present or not
  if (response.status == 404) {
    const splittedUri = uri.split('compress')

    if (splittedUri.length != 2) {
      callback(null, response)
      return
    }

    // NOTE: Parse the prefix, image name, imageWidth and format
    const path = splittedUri[1] // Read the required path (/480/uploads/123.jpg)
    const { originalKey, imageWidth, imageFormat } = getOriginalKey(path)

    if (!SUPPORTED_IMAGE_TYPES.some((type) => type == imageFormat.toLowerCase())) {
      response = responseUpdate(
        response,
        403,
        'Forbidden',
        'Unsupported image type',
        JSON_CONTENT_HEADER
      )
      callback(null, response)
      return
    }

    try {
      // NOTE: Get original image from S3
      const s3Object = await S3.getObject({ Bucket: BUCKET, Key: originalKey }).promise()

      if (s3Object.ContentLength == 0) {
        response = responseUpdate(
          response,
          404,
          'Not Found',
          'The image does not exist',
          JSON_CONTENT_HEADER
        )
        callback(null, response)
        return
      }

      // NOTE: Optimize the image
      let sharpObject = await Sharp(s3Object.Body)
      const metaData = await sharpObject.metadata()

      if (imageWidth < metaData.width) {
        sharpObject = await sharpObject.resize(imageWidth)
      }

      // >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
      // NOTE: The problematic line
      const quality = getQuality(path)
      // <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

      const compressedImageBuffer = await sharpObject.webp({ quality: QUALITY }).toBuffer()

      const byteLength = Buffer.byteLength(compressedImageBuffer, 'base64')
      if (byteLength == metaData.size) {
        callback(null, response)
        return
      }

      if (byteLength >= 1046528) {
        response = responseUpdate(
          response,
          400,
          'Invalid size',
          'The size of compressed image is too big',
          JSON_CONTENT_HEADER
        )
        callback(null, response)
        return
      }

      // NOTE: Generate a binary response with an optimized image
      response = responseUpdate(
        response,
        200,
        'OK',
        compressedImageBuffer.toString('base64'),
        WEBP_CONTENT_HEADER,
        'base64'
      )
      response.headers['cache-control'] = [{ key: 'cache-control', value: 'max-age=31536000' }]
    } catch (err) {
      console.error(err)
    }
  } else {
    headers['content-type'] = WEBP_CONTENT_HEADER
  }

  return response
}
renodesper
  • 25
  • 7
  • Can you please post the full error. Always post the error. – Jens Jun 08 '21 at 11:36
  • @Jens I already added the full error that was thrown by cloudfront. Please tell me if you need anything else. – renodesper Jun 09 '21 at 06:10
  • You should add the relevant CloudWatch log output of the execution. Start to end. – Jens Jun 09 '21 at 07:11
  • @Jens I already added the log from cloudfront. That is the only log that shows up since the lambda itself doesn't have any problem showing up in the log. For the lambda, I test it on the lambda console and it returns 200. – renodesper Jun 09 '21 at 07:34
  • We need the Lambda logs, not the Cloudfront logs. Every Lambda has CloudWatch logs. For Lambda@Edge it is more complicated, because you need to find in which region the Lambda was invoked in. It is probably the closest region to where you are. – Jens Jun 09 '21 at 07:49
  • @Jens Yes, you're correct. I just realized that even though the closest one is Singapore, the log somehow also exists in Sydney and Tokyo. I will try to check it again by myself first. Thank you for the valuable information. – renodesper Jun 10 '21 at 04:01

0 Answers0