0

I have the following scenario and am looking for an ideal strategy to manage memory for a go container running in Google Cloud Run. With the current design I'm reaching the Cloud Run memory limit of 512MiB resulting in a hard crash.

This service's main responsibility is to get a list of objects from a database, then use goroutines to create a task on a task queue for each object found in the db by calling Google Cloud Tasks endpoint to create the task.

The service will receive 36 requests to create tasks, and can run in parallel. A single instance is allowed to process up to 12 requests concurrently. In total there are over 500k objects in the source database, each needing a task to be created. I'm using gin to handle the requests, and Cloud Run environment variables to to control the following

  • Set max instances = 4
  • Set max concurrent requests = 12
  • Set max memory size of each instance = 512MiB
  • Set GOMEMLIMIT = 462MiB

Here is a simplified example of the service:

package example

import (
    "fmt"
    "github.com/gin-gonic/gin"
    "log"
    "net/http"
    "sync"
)

func main() {

    // SETUP GIN
    router := gin.Default()
    proxyErr := router.SetTrustedProxies(nil)
    if proxyErr != nil {
        log.Fatal(proxyErr)
    }
    router.POST("/users/tasks", userTasksHandler)
    runErr := router.Run(":8080")
    if runErr != nil {
        log.Fatal(runErr)
    }

}

func userTasksHandler(c *gin.Context) {
    // users will actually have tens or hundreds of thousands of items
    // The items come from a paginated service with 500 items per page
    users := []string {"item 1", "item 2", "item 3", "..."}

    var wgTasks sync.WaitGroup
    for _, v := range users{
        go createTask(&wgTasks, v )
    }
    wgTasks.Wait()
    c.String(http.StatusOK, "Done")
    return
}

func createTask(wgTasks *sync.WaitGroup, taskItem string){
    defer wgTasks.Done()
    fmt.Println("Creating task for %v\n", taskItem)
    // Create task in Google Cloud Task queue
    return
}

The container is built using the command:

gcloud run deploy --set-env-vars=GOMEMLIMIT=250MiB

Meaning I'm not specifying the dockerfile, Google's built service created that automatically.

I know by checking runtime.Version() that this is using go 1.20.4, so I'm certain the GOMEMLIMIT env variable is supported, although I wonder if it's taking effect.

Even if I set GOMEMLIMIT to something well below the 512MiB limit of Cloud Run, like 250MiB, it still uses more than 512MiB.

I'm able to keep it from hitting the memory limit by reducing "concurrency" of Cloud Run to 6 requests per instance, and increase the number of instances, but this doesn't seem very efficient.

My guess is that I probably need to build in some sort of limit to the number of goroutines that are created before new ones are added. But want to get opinions from the community on best practices for this type of workload.

Michael
  • 1,428
  • 3
  • 15
  • 34
  • 1
    `GOMEMLIMIT` is not a hard limit, it will not prevent you from allocating more memory than you have available. You can't just start a million goroutines and hope it all works out, you need to limit your own resource use. – JimB Jun 02 '23 at 19:02
  • 1
    If 6 concurrent requests is what it takes to stay within that memory limit, then that is your limit. If you want to look for more efficiency, you need to profile more carefully and figure out what can be made more efficient. – JimB Jun 02 '23 at 19:23
  • You need to determine if your container instances are exceeding the available memory. Look for related errors in the `varlog/system` logs. Note that in Cloud Run, files written to the local filesystem count towards the available memory. However, ff the instances are exceeding the available memory, you may want to consider increasing the memory limit. – Robert G Jun 02 '23 at 21:07
  • So I'm aware that it's not a hard limit, and do have hope it all works out, but in this case hope was not enough. However, I don't think concurrent sessions is the best way to handle this. Was able to find a way to limit the number of concurrent goroutines for each HTTP request running, and if the limit is exceeded it sleeps until the number goes down below a threshold of 2000. By doing that, I was able to raise max concurrent requests from 4 to 12 and at its peak reached 80% mem utilization instead of going over 100% like it did before. Although I wonder if this is the best approach. – Michael Jun 05 '23 at 18:56

0 Answers0