-2

I am having some problems with the concurrent HTTP connection in the golang. Kindly read the whole question, and as the actual code is quite long, I am using pseudocode

In short, I have to create a single API, which will internally call 5 other APIs, unify their response, and send them as a single response. I am using goroutines to call those 5 internal APIs along with timeout, and using channels to ensure that every goroutine has been completed, then I unify their response, and return the same.

Things are going fine when I do local testing, my response time is around 300ms, which is pretty good.

The problem arises when I do the locust load testing of 200 users, then my response time go as high as 7 8 sec. I am thinking it has to do with the HTTP client waiting for the resources as we are running a high number of goroutines.

like 1 API spin up 5 go-routine, so if each of 200 users makes API requests at the rate of supposing 5 req/sec. Then a total number of goroutines goes way higher. Again this is my assumption only

p.s. normally the API I am building is pretty good in response time, I am using all the caching and stuff and any response greater than 400ms should not be the case

So can anyone please tell me how can I tackle this problem of increasing response time when number of concurrent users increases


Locust test report enter image description here


pseudo code

simple route

group.POST("/test", controller.testHandler)

controller

type Worker struct {
    NumWorker int
    Data      chan structures.Placement
}

e := Worker{
    NumWorker: 5, // Number of worker goroutine(s)
    Data:      make(chan, 5) /* Buffer Size */),
}

//call the goroutines along with the 
for i := 0; i < e.NumWorker; i++ {

    // Do some fake work
    wg.Add(1)
    go ad.GetResponses(params ,chan ,  &wg) //making HHTP call and returning the response in the channel

}

for v := range resChan {
    //unifying all the response, and return the same as our response
    switch v.Tyoe{
    case A :
        finalResponse.A = v
    case B
        finalResponse.B  = v
    } 
}

return finalResponse



Request HTTP client

//i am using a global http client with custom transport , so that i can effectively use the resources
var client *http.Client

func init() {
    tr := &http.Transport{
        MaxIdleConnsPerHost: 1024,
        TLSHandshakeTimeout: 0 * time.Second,
    }

    tr.MaxIdleConns = 100
    tr.MaxConnsPerHost = 100
    tr.MaxIdleConnsPerHost = 100

    client = &http.Client{Transport: tr, Timeout: 10 * time.Second}
}



func GetResponses(params , chan  ,wg){
    res = client.Do(req)
    chan <- res

}

iron_man83
  • 67
  • 2
  • 9
  • 7
    It's unlikely that the http connections themselves are the bottleneck. There's not enough info here to even guess at what could be slowing the responses. – JimB Jan 19 '22 at 17:56
  • 4
    A load test only tells you how much load that operating context can handle; the machine your service is hosted on, the network it's connected to, and the same for the external endpoints your service is reaching out to. Maybe the performance you're seeing is great, maybe it's terrible, but without context, it's just a number that says nothing. Since you believe you've detected a performance problem, the first order of business would be to profile your application using `pprof`. – Adrian Jan 19 '22 at 18:39
  • 2
    You have to keep in mind that you system has hardware limits, if you have a CPU with for example 8 logical cores, you can only ever execute 8 goroutines in parallel. Assuming that 100% of the response time is spent on the CPU and not waiting for IO, at an response time of 300ms you can at most process ~27 requests per second. Any higher req/s and some requests will have to wait before there is time on the CPU. The equation changes if more time is spent on IO then CPU or when you have more cores. But it is normal to see higher latency with higher load – Dylan Reimerink Jan 19 '22 at 19:43
  • Track response times of your external dependencies as key performance indicators of your own. Your requests cannot respond until all dependent requests respond to you. – erik258 Jan 19 '22 at 21:11
  • @JimB can you please tell me what other info do you need for this problem. – iron_man83 Jan 20 '22 at 02:20

1 Answers1

1

So I have done some debugging and span monitoring , and turns out redis was the culprit in this. You can see this https://stackoverflow.com/a/70902382/9928176

To get an idea how I solved it

iron_man83
  • 67
  • 2
  • 9