How can I get a size of request.Header in bytes in Golang?

Question

I need to find the size of request.Header where request has *http.Request type:

req, err := http.NewRequest("GET", "/", nil)
cookie := &http.Cookie{Name: "foo", Value: "bar"}
req.AddCookie(cookie)

I tried

len(request.Header) # returned the number of elements in the map -- essentially the number of headers

and

for k, v := range req.Header {
  bytesSize += len(k) + len(v)
}

that didn't work either since v was a map.

I found Computing the memory footprint (or byte length) of a map question but the answer seems pretty complicated (and their map values are integers which is not the case here).

Update: actually here's the definition of type Header map[string][]string so we don't have to use recursion.

I'm curious as to why you'd need to compute the size (in bytes) of a map. If anything, that size is going to be implementation-dependent, and your code likely won't be very portable. — jub0bs, Oct 26 '21 at 23:02
You can use `Header.Write` to write it out to a buffer and see how large it is. But why would you need that? — Burak Serdar, Oct 26 '21 at 23:04
I'd like to implement https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/431. — Alex Kuzminov, Oct 26 '21 at 23:06

erik258 · Accepted Answer · 2021-10-27T02:25:47.793

https://pkg.go.dev/net/http#Server.MaxHeaderBytes can handle this for you.

This demo doesn't work reliably in the Playground (dial or connect timeouts) . It seems to work reliably locally though, which makes me guess it's an artifact of the playground's behavior.

We'll start an http Server with alow MaxHeaderBytes and then surpass it greatly.

package main

import (
    "context"
    "fmt"
    "io"
    "net"
    "net/http"
    "strings"
    "time"
)

func main() {
    res := make(chan error)
    mux := http.NewServeMux()
    mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "%+v", r.Header)
    })
    s := &http.Server{
        Addr:           "127.0.0.1:8103",
        Handler:        mux,
        ReadTimeout:    1 * time.Second,
        WriteTimeout:   1 * time.Second,
        MaxHeaderBytes: 2048,
    }
    if l, err := net.Listen("tcp", "127.0.0.1:8103"); err != nil {
        panic(fmt.Errorf("Couldn't listen: %w", err))
    } else {
        go func() {
            res <- s.Serve(l)
        }()
    }
    client := &http.Client{
        Timeout: 3 * time.Second,
    }
    req, err := http.NewRequest("GET", "http://127.0.0.1:8103", nil)
    if err != nil {
        panic(err)
    }
    req.Header.Add("X-Long-Header", strings.Repeat("long ", 2048)+"header")
    resp, err := client.Do(req)
    if err != nil {
        panic(fmt.Errorf("HTTP Request failed: %w", err))
    }
    fmt.Println(resp)
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        panic(fmt.Errorf("Could not read response body: %w", err))
    }
    fmt.Println("Body:", string(body))
    s.Shutdown(context.Background())
    <-res
}

Here, I'm setting MaxHeaderBytes to a fairly small value. I am passing far more than that value in my X-Long-Header: long long long .... header. If you can get the playground to work (just run it a few times) or run it locally, you'll get:

&{431 Request Header Fields Too Large 431 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8]] 0xc00001a180 -1 [] true false map[] 0xc000176000 <nil>}
Body: 431 Request Header Fields Too Large

As you can see, the 431 will automatically be generated if all headers are too large.

It might be appropriate for your handler itself to respond with a 431 if particular headers were too long, but by the time your handler has been passed an http.Request, the headers have been received. It doesn't make sense to try to compute the total length of the headers yourself and then respond with a 431 based on that.

Besides, standard headers may come and go, so it would be unwise to restrict overall header size too closely.

Instead, check whatever individual headers you're concerned about.

How can I get a size of request.Header in bytes in Golang?

1 Answers1