2

Using AWS I have set up a Lambda function using Python to act as a proxy. From the API Gateway I have linked the lambda (with 'Use Lambda Proxy integration' checkbox selected) to my API Gateway. Everything is working fine except that the max size of 6mb for the lambda function is getting in the way of my large files being returned successfully. The lambda prints an error that says "Response payload size exceeded maximum allowed payload size. Function.ResponseSizeTooLarge" and the API Gateway gets this message from lambda and prints 'Received response. Status 200', 'Lambda execution failed with status 200 due to customer function error: Response payload size exceeded maximum allowed payload size' and then API gateway spits out a 502 with the message 'Internal Server Error'.

What I would expect to happen is that API Gateway would see this error message and automatically return a 413 Request Entity Too Large so that i can react to that specific error status from my webapp.

In the past I have been able to use the Integration Response section of the api gateway to look for this error message and override the outgoing status code like so:

#set($inputRoot = $input.path('$'))
$input.json("$")
#if($inputRoot.toString().contains("ResponseSizeTooLarge"))
#set($context.responseOverride.status = 413)
#end

Unfortunately because i am using this lambda function from the api gateway as a lambda proxy I am unable to use the Integration Response to search for this specific message and override the outgoing status code.

Is there another way I can get the API Gateway to see that this 200 it is getting from the lambda actually has an error message and should be a 413 anyway? The only other way i can think to do this is check the payload size from the lambda before returning it to the API Gateway manually and then changing the status code before returning if its too large. I do not want to do this because the only way I have seen to get byte size of objects in python is by importing sys and using:

sys.getsizeof(lambdaResponse)

and that seems to only be returning a size of ~258 even though the actual size of its contents are much larger than that. I also do not want to be checking the size of every payload because it is probably adding unnecessary delay to responses that are not too large to be handled.

skyleguy
  • 979
  • 3
  • 19
  • 35

2 Answers2

1

As of now, there isn't a default way to achieve this. As you have called out, the best you can do is check the payload size if lambda itself and then return/throw exception.

Just a suggestion, if your use-case demands the response payload to exceed the allowed limit of the lambda function, I would suggest, store the response payload in some s3 bucket and return the s3 url of the payload as a response. This way, your application will scale well with any increase response paylaod. Moreover, if these responses are cachable, you would easily be able to re-use the same s3Url with slight modification in your application logic.

mango
  • 548
  • 3
  • 8
  • i do believe that this s3 solution would be the ultimate goal but as for now these responses cannot be put into s3. I will definitely keep this idea in mind when we get these responses migrated to cloud. As for now are you familiar with python's sys.getsizeof() method and how to allow it to return to me the size in bytes of any object I pass in? it seems to be unreliable in giving me the correct value in bytes of my lambda response. – skyleguy Apr 22 '20 at 21:00
  • 1
    @skyleguy, These response payloads will flow as JSON, that means you could just serialize the object into a string and calculate its length like: len(json_obj.encode("utf-8")). – mango Apr 23 '20 at 22:45
1

so while unfortunately there is no default way to handle this situation from the api gateway, there is still the option to handle this error in the lambda function. Before returning the response from the lambda to the api gateway I used the advice from mango and checked the byte size of the response by using:

len(json_obj.encode("utf-8"))

and if it was over the limit that lambda can return i changed the response.status to be 413 and changed the body to be an error message explaining what happened

skyleguy
  • 979
  • 3
  • 19
  • 35