3

I am using redigo with connection pool. Below is my code for setting up connection pool and the wrapper for SET call to Redis

//Create a pool of client connections to Redis
func newPool(MaxIdleConns int, Timeout int) *redis.Pool {
    return &redis.Pool{
        // Maximum number of idle connections in the pool.
        MaxIdle: MaxIdleConns*10,
        // Close connections after remaining idle for this duration
        IdleTimeout: time.Duration(Timeout) * time.Second,
        // Maximum number of connections allocated by the pool at a given time.
        MaxActive: 500,
        // If Wait is true and the pool is at the MaxActive limit, then Get() waits
        // for a connection to be returned to the pool before returning.
        Wait: true,
        // Dial is an application supplied function for creating and
        // configuring a connection.
        Dial: func() (redis.Conn, error) {
            // c, err := redis.DialTimeout("tcp", ":6379", 10*time.Second, 10*time.Second, 10*time.Second)
            c, err := redis.Dial("tcp", ":6379")
            if err != nil {
                fmt.Println("Redis Pool Dial error: ", err)
            }
            return c, err
        },
    }
}

//SET Wrapper
func SET(p *redis.Pool, key string, value []byte) error {
    c := p.Get()
    defer c.Close()
    _, err := c.Do("SET", key, value)
    return err
}

The above code leaks memory. I am closing the connection properly in defer c.Close(). Surprisingly if I comment out _, err := c.Do("SET", key, value) as in I don't do any c.Do() then the program doesn't leak memory.

Utkarsh
  • 171
  • 1
  • 14
  • 1
    What is the evidence that the program leaks memory? How many connections are in the pool at the time the leak is observed? – Charlie Tumahai Mar 26 '19 at 05:01
  • I did top on the program. The memory usage goes on increasing which doesn't happen without the Do. Almost 10,000 active connections. The code is making around 20,000 SET requests per second. – Utkarsh Mar 26 '19 at 05:06
  • 1
    A count of 10,000 active connections is huge! You should pipeline over a smaller number of connections. Does the memory stop growing after the application creates all of those connections? – Charlie Tumahai Mar 26 '19 at 05:10
  • No, I don't think so. It reached 32GB after which the process crashed. – Utkarsh Mar 26 '19 at 05:11
  • 1
    is `SET` being called inside a `go func`? or the `main` goroutine. – whitespace Mar 26 '19 at 10:57
  • 3
    Please provide an [mcve]. The idle timeout and max idle values use values not shown in the question. Show how SET is called. – Charlie Tumahai Mar 26 '19 at 12:33
  • Which redis package are you using? This doesn't look like go-redis, the one I'm using!! – Nannan AV Mar 26 '19 at 13:02
  • @NannanAV The question uses Redigo per the tag on the question. – Charlie Tumahai Mar 26 '19 at 16:43

0 Answers0