How to retry HTTP POST requests

Question

I have implemented the following code for retrying http post requests in Go.

In second retry attempt, I always get request body as null. I have tried defer req.body.close() but it is not working. Can anyone please help me this issue?

func httpRetry(req *http.Request, timeOut time.Duration, retryAttempts int, retryDelay time.Duration) (*http.Response, error) {

    attempts := 0

    for {
        attempts++
        fmt.Println("Attempt - ", attempts)
        statusCode := 0

        tr := &http.Transport{
            TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
        }
        var netClient = &http.Client{
            Timeout:   time.Second * timeOut,
            Transport: tr,
        }
        fmt.Println("ret 88888 ", req)
        response, err := netClient.Do(req)

        //defer req.Body.Close()

        fmt.Println(response, "  dd", err)

        if response != nil {
            fmt.Println(response.StatusCode)
            statusCode = response.StatusCode
        }

        if err != nil {
            fmt.Println(err)
            //return response, err
        }

        if err == nil && statusCode == http.StatusOK {
            return response, nil
        }

        if err == nil && response != nil {
            defer req.Body.Close()
        }

        retry := retryDecision(statusCode, attempts, retryAttempts)
        if !retry {
            return response, err
        }

        fmt.Println("Retry Attempt number", attempts)
        time.Sleep(retryDelay * time.Second)
        fmt.Println("After Delay")
    }
}

func retryDecision(responseStatusCode int, attempts int, retryAttempts int) bool {

    retry := false
    fmt.Println("Retry Decision 0 ", responseStatusCode, attempts, retryAttempts)
    errorCodeForRetry :=
        []int{http.StatusInternalServerError, http.StatusUnauthorized, http.StatusNotImplemented, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout}

    for _, code := range errorCodeForRetry {
        if code == responseStatusCode && attempts <= retryAttempts {
            retry = true
        }
    }

    fmt.Println("Retry Decision ", retry)

    return retry

}

Please find error details below

Attempt -  1
ret 88888  &{POST https://localhost:8080/logs/ HTTP/1.1 1 1 map[] {{"converstnId":"","boId":"","msgId":"","serviceName":"","headers":"","properties":"","message":"","body":"aa","exceptionMessage":"","logType":"","exceptionStackTrace":""}}

0x5ec890 170 [] false map[] map[] map[]
}

Attempt -  2
ret 88888  &{POST https://localhost:8080/logs/ HTTP/1.1 1 1 map[] {} 0x5ec890 170 [] false  map[] map[] <nil> map[]

}

your `defer req.Body.Close()` fired only then function returned not then your loop reruned. — Roman Kiselenko, Feb 15 '19 at 07:30
You can send a request only once. You have to create a new request for each try (or at the very least recreate the body). — Peter, Feb 15 '19 at 09:31
You shouldn't be creating a new `Transport` and `Client` for every loop iteration - in fact, most likely, you should be using the same `Transport` and `Client` instances throughout the entire application unless you actually need different configurations. — Adrian, Feb 15 '19 at 14:52

mkopriva · Accepted Answer · 2019-02-15T09:58:21.057

To be able to reuse a request whose body is non-nil, you first need to make sure that the body has been closed, not by you, but by the client's RoundTripper.

the relevant part of the docs:

RoundTrip must always close the body, including on errors, but depending on the implementation may do so in a separate goroutine even after RoundTrip returns. This means that callers wanting to reuse the body for subsequent requests must arrange to wait for the Close call before doing so.

When reusing requests that don't have a body, like GET, you still need to be careful:

If the request does not have a body, it can be reused as long as the caller does not mutate the Request until RoundTrip fails or the Response.Body is closed.

Taken from here.

So as long as you're sure that the RoundTripper closed the body, what you can do is to reset the body of the request at the top of each iteration. Something like this:

func httpRetry(req *http.Request, timeOut time.Duration, retryAttempts int, retryDelay time.Duration) (*http.Response, error) {

    // ...

    data, err := ioutil.ReadAll(req.Body)
    if err != nil {
        return nil, err
    }

    // close the original body, we don't need it anymore
    if err := req.Body.Close(); err != nil {
        return err
    }

    for {

        req.Body = ioutil.NopCloser(bytes.NewReader(data)) // reset the body

        // ... your code ...

    }

    // ...
}

Thanks @mkopriva. It works. I have made following changes to handle nil body.if req.Body != nil { data, err = ioutil.ReadAll(req.Body) if err != nil { data = []byte("") } fmt.Println("Input", string(data)) } // close the original body, we don't need it anymore if req.Body != nil { if err := req.Body.Close(); err != nil { return nil, err } } — Max2019, Feb 15 '19 at 12:07

How to retry HTTP POST requests

1 Answers1