Python 2 -> 3 Django migration causes field parameter type change

Question

We are transitioning a Django project from Django 1.8 -> 2.1 and Python 2.7 -> 3.6.

In the old project version, there are Django models that looked like this, for example:

# models.py

from django.db import models

class RowStatusModel(models.Model):
    active = models.BooleanField(default=True, db_column='is_active')
    # ...
    class Meta:
        abstract = True

Notice that from __future__ import unicode_literals is not used in this module. That means that db_column is a Python 2 str, corresponding to bytes in Python 3. The initial migration, 0001_initial.py, looks like this:

# 0001_initial.py

operations = [
    # ...
    ('row_ef', models.BooleanField(default=True, db_column=b'is_active')
    # ...
]

Notice the byte-literal b'is_active, which I suppose was done by Django in the interest of being more explicit, but am not sure.

Now after transitioning most of the codebase with 2to3 and running makemigrations, Python 3 treats a string literal as what would be the unicode type in Python 2, and consequently generates a migration where the db_column is a string literal, for every single model that inherits from RowStatusModel:

# migrations/0023_auto_20180827_1955.py 
migrations.AlterField(
    # ...
    field=models.BooleanField(default=True, db_column='is_active')
), # ...

What effect, if any, will this have on the database side when ./manage.py migrate is ran? Is the "change" purely on the Python side, or what side effects may be created?

Database engine is django.db.backends.postgresql.

I'm aware that we can just clone the RDS instance and revert back to that if migrate causes immediate problems, but I am more concerned about more subtle issues being introduced that would not be caught until much later on.

As noted in the last comment on that ticket, you can just edit the existing migrations to remove the `b` prefixes on strings. The newly generated migration will not have any effect if you choose to generate it. — solarissmoke, Aug 28 '18 at 12:24
@BradSolomon That's what we did when migrating to Python3 (altering existing migrations). No new migrations were generated and we had no issues (backends usually postgres). — user2390182, Aug 28 '18 at 12:28

Brad Solomon · Accepted Answer · 2019-02-06T13:47:11.700

12

I came across a Django ticket after asking this question, where the recommendation from a Django developer is to edit any legacy migrations files (such as 0001_initial) that contain (Python 3) bytes literals, removing the b and making them string-literals in Python 3.

Editing migrations to fix this issue is safe.

From there you should presumably be able to delete the migrations modules that were made with python3 ./manage.py makemigrations, and redo that command, which should no longer take issue with the type of that argument.

I used the following to do a mass-find-and-replace of all instances of 'db_column=b' with 'db_column='. Granted, you should absolutely check git diff before making that commit.

grep -nrl "db_column=b" apps | xargs sed -i "s/db_column=b/db_column=/g"

edited Feb 06 '19 at 13:47

answered Aug 28 '18 at 12:30

Brad Solomon

38,521
31
149
235

2

You can check the database effect of a migration with `sqlmigrate` command. This migrations will most probably have no effect on db. Django generates migrations even if the `choices` parameter for a field changes, without any db effect. And there are other cases for 'no-sql' migrations. – smido Aug 23 '19 at 13:09
1

In a more complicated situation, where editing all the existing migrations would be too time-consuming, it's worth considering to simply generate this migrations without effect just to make django happy, and continue as usual (commit, migrate,..). – smido Aug 23 '19 at 13:12

Python 2 -> 3 Django migration causes field parameter type change

1 Answers1