Programmatically check whether there are django south migrations that need to be deployed

Question

My deployment strategy looks like this (using Fabric):

create a new virtualenv
deploy new code in new virtualenv
show a maintenance page
copy the current db to new db
migrate new db
point new code to new db
symlink current virtualenv to new venv
restart services
remove maintenance page

I want to iterate fast. Now, most of the code changes do not contain migrations. Also, the db is growing, so there is much overhead created by copying the database everytime I deploy a (mostly small) change. To avoid copying the database I want to check whether there are migrations that need to be deployed (prior to step 4). If there are no migrations, I can go straight from step 2 to step 7. If there are, I will follow all the steps. For this, I need to check programmatically whether there are migrations that need to be deployed. How can I do this?

score 6 · Accepted Answer · edited Dec 25 '12 at 14:30

In step 2 while deploying the new code, you could deploy a script which when run on the server will detect if there are new migrations.

Example code is as follows:

# copied mostly from south.management.commands.migrate
from south import migration
from south.models import MigrationHistory

apps  = list(migration.all_migrations())

applied_migrations = MigrationHistory.objects.filter(app_name__in=[app.app_label() for app in apps])
applied_migrations = ['%s.%s' % (mi.app_name,mi.migration) for mi in applied_migrations]

num_new_migrations = 0
for app in apps:
    for migration in app:
        if migration.app_label() + "." + migration.name() not in applied_migrations:
            num_new_migrations = num_new_migrations + 1

return num_new_migrations

If you wrap the above code up in a script, your fabric deployment script can use the run operation to get the number of new migrations.

If this returns zero, then you can skip the steps associated with copying the database.

Thanks, Philip, for showing the code. This is almost exactly how I implemented it. Works great, have already done a lot of deployments with and without migrations using this method. — Sander Smits, Dec 13 '11 at 22:09

score 4 · Answer 2 · edited Dec 20 '17 at 08:56

4

 ./manage.py migrate --all --merge --list | grep "( )"

Will tell and show you which migrations. If you want a return code or count, use wc.

This has the advantages of not copying and pasting code like the accepted answer (violating DRY), and also if the internal south api changes your code will still work.

UPDATE:

Django 1.7 changed the output to use bracket instead of parenthesis and Django 1.8 introduced a showmigration command:

./manage.py showmigrations --list | grep '[ ]'

edited Dec 20 '17 at 08:56

Arnaud P

12,022
7
56
67

answered Mar 12 '14 at 11:44

dalore

5,594
1
36
38

1

simple and straightforward, i like this. – caesarsol Mar 18 '14 at 18:17
Just to be 100% clear - will this apply any unapplied migrations? Or will it simply list unapplied migrations? – Gershom Maes Jan 05 '16 at 16:55
It will only list and not apply anything. Perfectly safe. – dalore Jan 05 '16 at 20:19

score 2 · Answer 3 · answered Feb 13 '15 at 00:35

2

dalore's answer updated for Django 1.7+

./manage.py migrate  --list | grep "\[ ]"

If you just want the count then:

./manage.py migrate  --list | grep "\[ ]" | wc -l

answered Feb 13 '15 at 00:35

Jordi

886
11
11

score 1 · Answer 4 · answered Aug 17 '11 at 15:11

1

Why are you moving the databases around? The whole point of migrations is to apply the changes you made in development to your production database in place.

Your steps should really be:

create a new virtualenv
deploy new code in new virtualenv
show a maintenance page
migrate new db
symlink current virtualenv to new venv
restart services
remove maintenance page

And the migration step doesn't take that long if there's no actual new migrations to run. It'll just run through each app saying it's already up to date.

If you're copying the database to have a backup, that's something that should be running anyways on cron or something, not as part of your deployment.

Actually, I'm confused on the creating a new virtualenv each time too. The normalized (read: most typical) deployment is:

deploy new code
migrate db
restart services

If you want to add back in the maintenance page stuff, you can, but the process takes only a minute or two total.

answered Aug 17 '11 at 15:11

Chris Pratt

232,153
36
385
444

The purpose of my strategy is to be able to roll back fast. Rolling back with my strategy is just a matter of symlinking to the previous virtualenv which still points to the right db. With your suggestion, if something goes wrong, you're stuck with a migrated database and a virtualenv that is difficult to roll back. To roll back the db, you need a backed up database which is unlikely to be up to date. Also, rolling back with South is not recommended. I want to stick to my strategy, but without copying and migrating the database, if there are no migrations. So, my question still stands. – Sander Smits Aug 17 '11 at 17:26
Why is rolling back with South not recommended? If you set up your South migrations properly, such that data is never deleted, only moved around, then it's perfectly kosher to rollback a migration. That literally is the entire purpose of migrations. – Chris Pratt Aug 17 '11 at 20:22
I thought the author of South (Andrew Godwin) said that himself (Djangocon EU 2011). I might have misinterpreted it though. If there are no real issues with properly setup backwards migrations, I'll probably should go that way. – Sander Smits Aug 18 '11 at 06:15
On the other hand, a possible problem with using backward migrations is that they have to be absolutely right, otherwise they might themselves be a possible point of failure. You might end up with a wrongly backwards migrated database. Since I'm trying to automate as much as possible, not having to check and test backwards migrations is a huge advantage. – Sander Smits Aug 18 '11 at 06:36
I could see that argument, but I'd argue that it's just as important to have a good backwards migration as anything else. There's more reasons to rollback a migration than just "something doesn't work right". It's a form of version control for your DB structure, and if you want to rollback your source to a particular revision when an model was in a better state for going forward than it is now, having good backwards migrations will be a lifesaver. Additionally, you may not always be the one managing the system. If some later developer comes expecting migrations to work, your system is borked. – Chris Pratt Aug 18 '11 at 14:49
Inspired by a South talk yesterday on Djangocon US by Brian Loft, I decided to go the way I stated in my original question. Basically, I wrote a utility function that checks whether there are unapplied migrations. Rolling back is still just a matter of pointing a symlink to the previous virtualenv, while at the same time not having the cost of cloning the database when there are no migrations that need to be applied. – Sander Smits Sep 08 '11 at 18:16

Programmatically check whether there are django south migrations that need to be deployed

4 Answers4

Linked