2

We have an express server that runs on google cloud run. When our server attempts to respond with a HTTP status code of 204 this gets transformed into 502 Bad Gateway. I have confirmed that this is only happening in GCP. In this specific example, we are using Google's fully managed endpoint. Running this on my local machine yields the expected results. Also, sending a 200 status code works successfully with the same code.

Error code: upstream connect error or disconnect/reset before headers. reset reason: protocol error

Here is our express endpoint that responds with the 204 status code:

app.post('/', [cors(corsConfig), jsonParser], async (req, res) => {

  res.setTimeout(timeout, () => {
    log(’timeout’)
    res.sendStatus('204');
    res.end();
  });

});

Connection details:

  • Utilizing Google's managed endpoint
  • HTTP/2 disabled

here is our Dockerfile:

# Use the official lightweight Node.js 12 image.
# https://hub.docker.com/_/node
FROM node:12-slim AS base

# Create and change to the app directory.
WORKDIR /usr/src/app

# Run our webpack build inside a separate stage
FROM base AS build

# Copy application dependency manifests to the container image.
# A wildcard is used to ensure both package.json AND package-lock.json are copied.
# Copying this separately prevents re-running npm install on every code change.
COPY package*.json ./
COPY yarn.lock ./

# Install all dependencies for a build
RUN yarn

COPY . ./
RUN yarn build

FROM base AS runtime

COPY package*.json ./
COPY yarn.lock ./

# Copy only the built app output
COPY --from=build /usr/src/app/dist ./dist

# Re-install only production dependencies, to reduce image size ($$)
RUN yarn --only=production

COPY public ./public
COPY partners.yml ./

# Run the web service on container startup.
CMD [ "yarn", "start" ]

1 Answers1

3

We were having the same issue with a Django application in Google Cloud Run. The issue was that our application was not sending an empty response body as is required with a 204 status. The framework we are using added some wrapping around our empty result.

Many servers incorrectly send a body in a 204 response, so browsers generally use heuristics to see if a response body is present (link). The Cloud Run proxy appears to adhere to the standard strictly, which results in an HTTP protocol error when a response body is present.

To fix it you need to make sure you return an actually empty response from your application, or just use a 200 status code.

JanKanis
  • 6,346
  • 5
  • 38
  • 42
  • What framework were you using and what wrapper was it adding? I'm using DRF and even when I was sending back an empty string, it was still causing a 502. I had to resort to returning an HTTP 200 instead of 204 to fix this. – kevlar Jun 13 '23 at 22:45
  • Turned out I had to return None in the response using the 204, not an empty string. – kevlar Jun 13 '23 at 22:58
  • If I remember correctly, it was Django REST framework that wrapped whatever our code was returning in a JSON object. – JanKanis Jun 14 '23 at 09:26