35

If I have a non-nullable model field, remove it, and create a migration, that migration becomes non-reversible:

Consider the following model:

class Foo(models.Model):
    bar = models.TextField()
    test = models.TextField()  # This field is to go away, bye-bye!

And migration:

# app/migrations/003_remove_foo_test.py

class Migration(migrations.Migration):

    dependencies = [
        ('app', '0002_foo_test'),
    ]

    operations = [
        migrations.RemoveField(
            model_name='foo',
            name='test',
        ),
    ]

Unapplying this migration throws an exception:

$ src/manage.py migrate app 0002
Operations to perform:
  Target specific migration: 0002_foo_test, from app
Running migrations:
  Unapplying app.0003_remove_foo_test...Traceback (most recent call last):
...
django.db.utils.IntegrityError: column "test" contains null values

Of course, this is the expected behaviour, it is clearly documented and I'm not asking why this happens:

Bear in mind that when reversed this is actually adding a field to a model; if the field is not nullable this may make this operation irreversible (apart from any data loss, which of course is irreversible).

However, we all make mistakes and sometimes we just need to somehow reverse a field deletion, even if that means manually providing an ad hoc stub value for all reversed non-null fields. For instance, South migrations optionally allow the reversal of such operations (by asking the developer on whether to provide a default for restored fields, or disallow reverse migration), which doesn't seem to be the case with the all new fancy Django 1.7 migrations.

Question: what is the easiest/fastest way to undo a field removal with Django 1.7+ migrations (assuming it has already happened)? It doesn't necessarily need to be fully scripted with Python, a set of manual instructions will do.

ferrangb
  • 2,012
  • 2
  • 20
  • 34
Ilya Semenov
  • 7,874
  • 3
  • 30
  • 30
  • Why it raises `IntegrityError` instead of `django.db.migrations.exceptions.IrreversibleError`? Was there any discussion about this? – Qback Jun 12 '23 at 11:47

4 Answers4

31

You can manually edit your migration and add AlterField with default value for a field just before RemoveField. It should be safe even after applying migration. That will make RemoveField that will happen after to be reversible.

An example. Having field in model summary named profit that was defined before removal like that:

profit = models.PositiveIntegerField(verbose_name='profits')

you should add before RemoveField of it an AlterField like that:

migrations.AlterField(
    model_name='summary',
    name='profit',
    field=models.PositiveIntegerField(verbose_name='profits', default=0),
    preserve_default=False,
    ),
GwynBleidD
  • 20,081
  • 5
  • 46
  • 77
  • `AlterField` doesn't accept `preserve_default` argument (but it doesn't seem to be needed in this case, as `default` doesn't get its way to the database schema anyway). Other than that, this seems to be a correct and optimal solution indeed. Thanks! – Ilya Semenov Mar 06 '15 at 17:58
  • According to [documentation](https://docs.djangoproject.com/en/1.7/ref/migration-operations/#django.db.migrations.operations.AlterField), yes it does, but it was added in 1.7.1. And in that case, preserve_default shouldn't matter. It does get its way to the database, but only for a short while. – GwynBleidD Mar 07 '15 at 18:16
  • 11
    I was able to get this working in Django 1.8 with `preserve_default=True`. – Jess Oct 05 '16 at 22:15
  • @emyller I added RunPythons to create and remove a dummy object in the related model table so that the default value would point to something, it worked :) – Ariel Dec 19 '18 at 17:20
  • I tried to use `AlterField` to make the field nullable before deleting it (instead of providing a default) but it didn't work. Any idea why? I mean, how does Django figure out that a field that doesn't exist in the database was not nullable if not by looking at the modifications in the migration files? There must be a way I could force the field to nullable before the `RemoveField` operation is executed during a reverse migration. – Ariel Dec 19 '18 at 17:22
  • Yes, django is looking at migrations, but... You cannot import models directly from `models.py` files of your apps in migration! Instead use `apps.get_model` from `apps` object that is passed as first argument to function provided in `RunPython`. – GwynBleidD Dec 20 '18 at 10:42
  • I'm getting an integrity error because the field is unique. Is there any way to have a dynamic default? – Ariel Aug 13 '20 at 14:17
  • 1
    Did not work with `preserve_default=False`, but it did work without it. – ruohola Jul 22 '21 at 08:37
5

If you're trying to make future migrations reversible, you could try removing the field as three migrations.

  1. Make the field nullable
  2. Data migration. Forward, don't do anything. Backwards, convert nulls to a stub value.
  3. Remove the field

Each of these three steps should be reversible.

If you have already run the migration and need to reverse it, you could

  1. manually add the field, allowing nulls
  2. convert nulls to a stub value
  3. manually add the not null constraint
  4. Migrate with --fake to the previous migration
Alasdair
  • 298,606
  • 55
  • 578
  • 516
  • I'm not quite sure about "manually add the field" in Step 1. I'd surely like to avoid manually running ALTER TABLE .. ADD COLUMN, that's what the ORM layer always does and always should do. It's not up to the programmer to repeat all the little tricks the ORM does (naming fields, managing indexes, etc.) – Ilya Semenov Mar 05 '15 at 13:21
  • Manually running SQL is not ideal, but sometimes you might not have any other options. Somebody else might have a better suggestion. You might be able to get Django to generate the SQL with `./manage.py sqlmigrate --backwards`. – Alasdair Mar 05 '15 at 13:28
  • I can confirm that `./manage.py sqlmigrate --backwards` gives the correct SQL to do this migration, and that this isn't painful to run if you're using an IDE like PyCharm; the migration-editing approach did not work for me on a backwards migration. – Symmetric Jan 15 '16 at 01:37
1

The easiest way may be to use migrations.RunSQL

You can edit the migration so your operations list would look like this:

operations = [
    sql=[('alter table foo_test drop test)],
    reverse_sql=[('alter table foo_test add test varchar)]
]

That would be a hacky solution, but so would be probably any other.

David Buck
  • 3,752
  • 35
  • 31
  • 35
Jovan V
  • 130
  • 7
1

Just add 'default' and 'preserve_default' to your AddField or AddModel in older migration, Django will already know that it must recreate the column with the provided default

This Helped me

Moksh Modi
  • 101
  • 2
  • 5