configure echo http server to control transfer encoding

Question

I have a simple HTTP server written in golang using echo v4. When the response size is bigger than a certain size (threshold is 2.12K as I have tested), server sets the Transfer-Encoding header to chunked and sends the response in multiple chunks, but for smaller responses, the server does not set the Transfer-Encoding header and sends the response body plain.

I want to control this behavior, so that I can define the threshold where the echo HTTP server starts to chunk the response body. My google searches show that I can obtain this by manipulating the ResponseWriter, but I could not figure out how to do that in my code.

This is my code:

func main() {
    e := echo.New()

    e.Use(middleware.Logger())
    e.Use(middleware.Recover())

    e.GET("/hi", func(c echo.Context) error {
        data := make(map[int]string)
    
        for i := 0; i < repeatCount; i++ {
            data[i] = strings.Repeat("z", i)
        }
    
        return c.JSON(http.StatusOK, data)
    })
    
    e.Logger.Fatal(e.Start(":3000"))
}

Can anyone please help me here?

Try to implement own middleware and marshal the response, take the size of a byte array and put manually as a Content-length header and send the bytes as a response directly to the client usings write stream on the context, without calling next() https://echo.labstack.com/cookbook/middleware/#how-to-write-a-custom-middleware — Alexander R., Aug 28 '22 at 15:41
See question [Disable chunked transfer encoding](https://stackoverflow.com/q/34794647/5728991). Execute this line before writing to the response: `c.Response().Writer.Header().Set("Transfer-Encoding", "identity")` — Charlie Tumahai, Aug 28 '22 at 16:52
@CeriseLimón can you please explain me why there is no Content-Length header in the response and how client can determine the end of the response when Transfer-Encoding is set to identity? — Zeinab Abbasimazar, Sep 06 '22 at 11:21

Charlie Tumahai · Answer 1 · 2022-09-06T16:16:15.253

The net/http server uses the identify transfer encoding when Content-Length header is set. If the length of the response body is known, set the content length header before writing to the response:

  w := c.Response().Writer
  w.Header().Set("Content-Length", strconv.Atoi(contentLength))

The application can buffer the response to compute the content length:

data := make(map[int]string)
for i := 0; i < repeatCount; i++ {
    data[i] = strings.Repeat("z", i)
}
p, err := json.Marshal(data)
// <-- insert code to handle error here
w := c.Response().Writer
w.Header().Set("Content-Length", strconv.Atoi(len(p)))
return c.JSONBlob(http.StatusOK, p)

The net/http server uses the identity transfer encoding when the Transfer-Encoding header is set to identity. The server terminates the response body by closing the connection. The client reads the response body to EOF. Set the header before writing to the response.

  w := c.Response().Writer
  w.Header().Set("Transfer-Encoding", "identity")

The net/http server buffers 2048 bytes of the response body. The application cannot change the size of this buffer. If the entire response body fits within this buffer, the server sets the Content-Length header and uses the identity transfer encoding.

Otherwise, the server uses chunked encoding.

Use an application controlled buffer to change the threshold at which chunking occurs:

w := c.Response().Writer
b := bufio.NewWriterSize(c.Response(), threshold)
w.Header().Set("Content-Type", echo.MIMEApplicationJSONCharsetUTF8)
err := json.NewEncoder(c.Response).Encode(data)
// <-- insert code to handle error here.

// If no data was written to the net/http server's reponse
// writer, then the entire response is in the bufio.Writer.
// Set the Content-Length header to the length of the buffered
// data.
if !c.Response().Committed {
   w.Header().Set("Content-Length", strconv.Atoi(b.Buffered())
}

// In all cases, flush bufio.Writer.
b.Flush()

configure echo http server to control transfer encoding

1 Answers1