Go: Run test from multiple package with DB initialization

Question

I have a GO project with this project structure (multiple couples of this kind of files in each package).

- api
    - userHandler.go
    - userHandler_test.go
- database
    - user.go
    - user_test.go

Inside user.go I have the User struct and the functions to Create/Get/Update a User (I'm using GORM but this is not the issue). In the user_test.go.

I'd like to have the DB cleaned (with all the data removed or in a certain state) for each different file, so I've tried to create 1 suite (using Testify) for each file, then use the SetupSuite function but the behaviour seems not deterministic, and probably I'm doing something wrong.

So my questions are:

Which is the best way to have a DB connection shared? Using a global variable is the best option?
Which is the best way to create the tables in the DB once and then init the DB with custom data before each file_test.go is run?

Right now I'm also having a strange bug: running

go test path/package1
go test path/package2

Everything works fine, but if I run (for testing all the packages)

cd path && go test ./...

I have errors that seems not to be deterministic, that's why I'm guessing that the DB connection is not handled properly

I'd make the connections and methods that interact with the db globalized at your testing package level and in then do the set up and tear down in each individual test file like user_test.go. — evanmcdonnal, Jun 25 '15 at 22:13
By it I mean seed the database in setup and delete those records in tear down. It's gotta be on a test by test basis cause that's the context where the relevant information is. — evanmcdonnal, Jun 25 '15 at 22:15
thanks, I've also updated my question because I have a strange issue with multiple packages, that's why I guess that the DB connection is not handled properly — Alessio, Jun 25 '15 at 22:22
Put the db stuff in a common testing package and import within the test files in those other packages. The test package should probably be at the level as package1 and 2 in your example. — evanmcdonnal, Jun 25 '15 at 22:24
I don't have a test package but I'm following the test_file.go convention so I have test file close to each source file. — Alessio, Jun 25 '15 at 22:25
Yes I'm saying create a test package in your project and implement common helper methods there. Import that package in each of those test files so that you can reuse the code which touches your data store. You want to keep the IO general and do the actions in those specific places so you can easily compare the expected and yielded results. — evanmcdonnal, Jun 25 '15 at 22:29

score 1 · Accepted Answer · answered Jun 26 '15 at 01:37

If your api package depends on your database package (which it appears to) then your api package should have a way to provide a database connection pool (e.g. a *sql.DB) to it.

In your tests for the api package, you should just pass in an initialised pool (perhaps with the test schema/fixtures pre-populated) that you can use. This can either be a global you initialise in init() for the api package or a setup() and defer teardown() pattern in each test function.

Here's the former (simplest) approach where you just create a shared database and schema for your tests to use.

package database

import testing

var testDB *sql.DB

// This gets run before your actual test functions do.
func init() {
    var err error
    db, err = sql.Open(...)
    if err != nil {
        log.Fatalf("test init failed: %s", err)
    }

    _, err := db.Exec(`CREATE TABLE ....`)
    if err != nil {
        log.Fatalf("test schema creation failed: %s", err)
    }
}

Some tips:

You can also call a setup() function can create a table with a random suffix and insert your test data so that your tests don't use the same test table (and therefore risk conflicting or relying on each other). Capture that table name and dump it in your defer teardown() function.
https://medium.com/@benbjohnson/structuring-applications-in-go-3b04be4ff091 is worth reading for some additional perspective.

Thanks, I did an it works. The only issue I have is the one I've described at the bottom of my open post, do you have any idea of what's happening? — Alessio, Jun 26 '15 at 08:01
What errors are you getting? Note that tests can run from a temp directory and cause problems if you use relative paths. — elithrar, Jun 26 '15 at 09:03
the errors are like "(Error 1005: Can't create table 'myschema.#sql-353_166a' (errno: 150)) or '(Error 1050: Table 'device' already exists) '. They seems not to be deterministic that's why I think that the test are run in parallel and not sequentially as I imagined (running go test ./... from my project root folder). — Alessio, Jun 26 '15 at 09:25
I've probably found the error: i was calling multiple time the Open() method, and this is not recommended. — Alessio, Jun 26 '15 at 13:56
Correct - Open creates a connection pool with multiple usable connections. There's typically no need to have more than one pool per app. — elithrar, Jun 26 '15 at 15:26

Go: Run test from multiple package with DB initialization

1 Answers1