2

I have recently written some Ecto migrations in a Phoenix application.
However, I always tested the migrations manually.

As our migrations grow more complex and need to alter data on our live system, I'd really like to write automated tests before deploying them.

Automatic tests would require:

  1. Reset the database to a point right before the migration
  2. Run migration older than the migration to be tested
  3. Prepare data and insert it to the database
  4. Run migration
  5. Verify results match expectations
  6. Cleanup database to ensure upcoming tests don't clash with the data

We're using ExUnit and ESpec, but I haven't found a way to only apply some migrations prior specific tests.

Is there a way to write automatic tests for Ecto migrations?

apaderno
  • 28,547
  • 16
  • 75
  • 90
Mene
  • 3,739
  • 21
  • 40

2 Answers2

3

This is possible with a bit of manual work. All these tests should have a tag (e. g. @tag :migr) and should be run as a separate test via 

mix test --only migr

Details.

The standard tests should be run via

mix test --exclude migr

The latter might be set as a default config, for the former I would create mix alias.

Then you should create your own task, similar to ecto.migrate. It would roll up all the migrations till the specified one (this might be passed as a parameter,) and perform tests you want.

Sidenote: the necessity to test migrations is a very bad sign of the code design in general.

Aleksei Matiushkin
  • 119,336
  • 10
  • 100
  • 160
  • Why is it a sign of bad code design? We had external events that forced us to change a portion of our data as well as simple data becoming more complex as features evolved, which resulted in a non trivial migrations. Most migrations are trivial, but some contain logic I'd prefer to test before applying them to out prod data. – Mene Oct 03 '19 at 08:36
  • 2
    It does probably mean that the database is not backward compatible. Also, the business logic should not belong to migrations; it should belong to the application. The application itself should be happy to run on both old and new version. – Aleksei Matiushkin Oct 03 '19 at 08:42
  • I agree for most cases, but at least in my current case we need to migrate data only for a temporal window of ~1 week because the business model has changed at a certain point in time. It's foreseeable that this will happen again in the future. Handling this in the application would mean a new special case every time that happens (our customers SHOULD not do these changes for past events but they insist) so I thought a migration would turn this to a one-time event and afterwards all data can be treated equally. – Mene Oct 03 '19 at 09:00
  • 1
    Well, maybe. It’s extremely hard to tell without a complete understanding of the use case but in general, this kind of migration surely might be avoided. – Aleksei Matiushkin Oct 03 '19 at 09:09
3

I would recommend against putting much logic into migrations. Migrations should change the structure of data layer, not the data itself.

The more direct approach to your issue is to create separate module with data transformation logic and write tests for this module. Then, you can apply migrations and call your module separately to apply transformations.

Also, keep in mind your application should work even if you decide to rollback application to previous release. I.e. if you want to remove column, create new column first, then transfer data and switch your application to use new column. On next deploy you can remove an old column.

You can call your data transformation module via Mix.Task.

achempion
  • 794
  • 6
  • 17
  • 1
    Thanks, this sounds similar to what @aleksei-matiushkin said. What I find appalling about this approach is that I will keep on testing the migration logic, as it has now become part of my test suite, but there is no need to continually test it, as there can be no regression. It is (usually) executed once and that's it. We currently have ~150 migrations and running the test suite already takes >15min. But maybe a combination of both ideas is what I go for: move logic to own module, tag and exclude tests by default. – Mene Oct 07 '19 at 14:03