golang resource ownership pattern (files, connections, close-ables)

Question

What is the proper way to manage resource ownership in golang? Suppose I have the following:

db, err := sql.Open("mysql", "role@/test_db")
am := NewResourceManager(db)
am.DoWork()
db.Close()

Is it typical to always have the calling function maintain ownership and responsibility for closing resources? This feels a little weird to me because after closing, am still retains a reference and could try to use db if I or someone else is not careful later on (I guess this is a case for defer; however, if I want to pass the ResourceManager am back from this block, how would I even defer the closing of the file properly? I actually want it to stay open when this block finishes execution). I find that in other languages I often want to allow the instance to manage the resource and then clean it up when it's destructor is called, like this toy python example:

class Writer():
   def __init__(self, filename):
       self.f = open(filename, 'w+')

   def __del__(self):
       self.f.close()

   def write(value):
       self.f.write(value)

Unfortunately, there are no destructors in golang. I'm not sure how I would do this in go other than something like this:

type ResourceManager interface {
   DoWork()
   // Close() ?
}

type resourceManager struct {
  db *sql.DB
}

func NewResourceManager(db *sql.DB) ResourceManager {
  return &resourceManager{db}
} 

db, err := sql.Open("mysql", "role@/test_db")
am := NewResourceManager(db)
am.DoWork()
am.Close()  // using method shortening

But this seems less transparent, and I'm not sure how to communicate that the ResourceManager also needs to be Close()'d now. I'm finding this a frequent stumbling block, i.e. I also want to have a resource manager that holds a gRPC client connection, and if these types of resources aren't managed by resource managing objects, it seems like my main function is going to be cluttered with a lot of resource management, i.e. opening and closing. For instance, I could imagine a case where I wouldn't want main to know anything about the object and it's resources:

...
func NewResourceManager() ResourceManager {
  db, err := sql.Open("mysql", "role@/test_db")
  return &resourceManager{db}
}
...
// main elsewhere
am := NewResourceManager()
am.DoWork()

This question is broad enough that the only correct answer is "it depends". — Adrian, Nov 26 '19 at 14:53

score 5 · Accepted Answer · edited Jun 20 '20 at 09:12

You chose a bad example, since you generally would reuse a database connection, instead of opening and closing one for each use. Hence, you would pass the db connection to functions using it and do the resource management in the caller without any need for an resource manager:

// Imports etc omitted for the sake of readability

func PingHandler(db *sql.DB) http.Handler (
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
       if err := db.ping(); err != nil {
          http.Error(w,e.Error(),500)
       }
    })
)

func main(){
    db,_ := sql.Open("superdb",os.Getenv("APP_DBURL"))

    // Note the db connection will only be closed if main exits.
    defer db.Close()

    // Setup the server
    http.Handle("/ping", PingHandler(db))
    server := &http.Server{Addr: ":8080"}

    // Create a channel for listening on SIGINT, -TERM and -QUIT
    stop := make(chan os.Signal, 1)

    // Register channel to be notified on said signals
    signal.Notify(stop, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)

    go func(){
            // When we get the signal...
            <- stop
            ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
            // ... we gracefully shut down the server.
            // That ensures that no new connections, which potentially
            // would use our db connection, are accepted.
            if err := server.Shutdown(ctx); err != nil {
                // handle err
            }
    }

    // This blocks until the server is shut down.
    // AFTER it is shut down, main exits, the deferred calls are executed.
    // In this case, the database connection is closed.
    // And it is closed only after the last handler call which uses the connection is finished.
    // Mission accomplished.
    server.ListenAndServe()
}

So in this example, there is no need for a resource manager and I honestly can not think of an example actually needing one. In the rare cases I needed something akin to one, I used sync.Pool.

However, for gRPC client connections, there is no need to maintain a pool, either:

[...]However, the ClientConn should manage the connections itself, so if a connection is broken, it will reconnect automatically. And if you have multiple backends, it's possible to connect to multiple of them and load balance between them. [...]

So the same principle applies: Create a connection (pool), pass it around as required, ensure that it is closed after all work is done.

Go proverb:

Clear is better than clever.

– Robert Pike

Instead of hiding resource management somewhere else, manage the resources as closely to the code where they are used and as explicitly as you possibly can.

golang resource ownership pattern (files, connections, close-ables)

1 Answers1