FactoryGirl and DatabaseCleaner best practices for entire test suite using integration and unit tests

Question

I have a Rails test suite that uses DatabaseCleaner, FactoryGirl, and Capybara-Webkit. All my tests pass on my machine when running the suite, including when running them all in parallel (parallel_rspec).

When I push to CI (circleci) I always get a failure in my integration tests. I suspect it is because I am using transaction strategy in my unit tests (controllers, services, views, etc...). Perhaps it is a bad practice, but if I need a model in my controller (for example) I have been doing FactoryGirl.create(:my_model). But I suspect every single place I use FactoryGirl to create a model I should also be making that test use a truncation strategy in DatabaseCleaner.

I just marked my entire suite to use truncation and pushed to CI and it's green which is great, but now the suite takes 50% longer to run.

So, when instantiating ActiveRecord models with FactoryGirl, should I ALWAYS mark those tests to use truncation?

Thank you for any and all feedback.

Answer 1

Test that need to be marked to use truncation (or deletion) are tests that access the database through multiple connections. When using Capybara for feature tests that's generally any tests using a driver other than rack-test, since they run the app in a separate thread which gets it's own connection to the database, and is handled by the recommended database_cleaner config - https://github.com/DatabaseCleaner/database_cleaner#rspec-with-capybara-example (the use of append_after for the block that starts the clean is very important)

Other types of tests generally don't create multiple threads or connections so truncation usually isn't necessary for them (and data needed in those tests is often built using the FactoryGirl build_stubbed method with find mocked to return the stubbed object and bypass the database completely).

Additionally if the tests pass on your machine they should pass on CI since I assume you're using the same database cleaner method on your machine as on CI, so I would guess it's more of a timing issue.

Answer 2

Also with DatabaseCleaner, once you set the method in a CI container, it's set that way for the entire run, so it's good practice to run your unit tests in a separate container than your features/system tests.