16

I'm using API Gateway Lambda proxy integration and trying to return a binary application/protobuf response. No matter what I do, the response body is always a base64 encoded string

  • I have application/protobuf setup as a binary media types in APIG
  • My client (javascript) is sending following headers in the POST: Accept: application/protobuf Content-Type: application/protobuf
  • My lambda is responing with content-type: application/protobuf, and correctly setting the IsBase64Encoded Lambda response to true

How do you get APIG to base64 decode the string? I swear I had this working a few months ago when I 1st tried this.

Note: I've also tried */* as a binary media types

Some related posts to add background:

Update:

Turns out I can only get it working if binary media type is set to */*. The client Accept header has no impact once it is set to this.

Many bad side effects of using */* because every response is attempted to get decoded (even when IsBase64Encoded is false or not set)

I thought it wasn't decoding because Chrome network inspect tools will always show binary data as base64 encoded in the Preview tab. You can see the protobuf in the Response tab.

rynop
  • 50,086
  • 26
  • 101
  • 112
  • You mentioned "many bad side effects of using */*", but I don't see them. Can you help me understand how to observe these side effects. I have a simple repo at https://github.com/activescott/apig-lambda-proxy-binary-response that demonstrates this but I'm not seeing any side-effects (e.g. on the html function response for example). – Scott Willeke Dec 17 '18 at 03:32
  • I have to admit, I didn't test the `*/*` on responses that were not binary. I was just making an educated guess that if APIG was configured to this, it would try to base64 decode EVERY response. Now that you mention this, I'm wondering if APIG only does the decode if `IsBase64Encoded` is true. However that doesn't make sense in my head. If this were the case why the need for APIG `binary media type` in the 1st place. – rynop Dec 17 '18 at 17:12
  • Thanks for responding. I'm trying to reason through the same things here :) And I have to admit, I'm unclear on the need/purpose of needing both `isBase64Encoded` _and_ `binaryMediaTypes`. I'll probably keep tinkering and update that repo with more details as I confirm them... – Scott Willeke Dec 18 '18 at 01:50
  • 1
    In case anyone might find it helpful, I have dedicated a repo at https://github.com/activescott/apigateway-lambda-binary-response to documenting and demonstrating the various details of binary responses with Amazon API Gateway + AWS Lambda. – Scott Willeke Dec 18 '18 at 07:22

1 Answers1

20

The problem was I'm using CloudFront in front of API Gateway, and I was not passing the Accept header to the origin (APIG).

The docs on handling binary with Lambda proxy are not great, so here is a quick summary:

  • Your client must send an Accept header who's 1st media type matches what you have set as a binary media types in API Gateway
  • Your Lambda, if serving a binary media type, must set IsBase64Encoded to true AND the body must be base64 encoded

If the clients Accept header matches an entry in API Gateway's binary media types and these conditions are met, API Gateway will transform (base64 decode) before sending a response to the client.

This blog post walks you through step-by-step on how to get it working (without CloudFront).

This is a full blown aws-blueprint for getting a production grade ci/cd with CloudFront.

rynop
  • 50,086
  • 26
  • 101
  • 112