Django - Adding a primary key to an existing model with data

Question

This is the existing model in a djnago app:

class Task_Master_Data(models.Model):
    project_id = models.ForeignKey(Project_Master_Data, on_delete=models.CASCADE)
    task_created_at = models.DateTimeField(auto_now_add=True)
    task_updated_at = models.DateTimeField(auto_now=True)
    task_name = models.CharField(max_length=200, null=True)

I want to add a new field (which will be a primary key):

    task_id = models.AutoField(primary_key=True, null=False)

When I am making migrations from the terminal, it will provide me with the option of adding a default value, but it will, understandably, bring an error of:

UNIQUE constraint failed: new__main_task_master_data.task_id

What would be the best way forward for this kind of a scenario without having to delete all the data.

Note, I am using the default sqlite3 database.

Answer 1

This is the result of an old bug that is still unfixed. See here for the ticket.

I have created a workaround.

First create an empty migration (change taskapp to the name of the app where your models.py with Task_Master_Data lives):

python manage.py makemigrations taskapp --empty

Then copy the below migrations into this file and run python manage.py migrate but be sure to adjust the name of the app (replace taskapp with the name of your app) and the dependencies (0010_task_master_data should be replaced by the name of the migration that comes before it).

First the existing rows in the database need values for the new task_id field. I solved this by creating an IntegerField with null=True, which I then manually fill using a RunPython command. I then remove the id field (my previous primary key), and use AlterField to change the field into the AutoField. A default value is required, but it should never be used. When you create a new Task_Master_Data the task_id should be auto-incrementing.

# Generated by Django 3.0.4 on 2022-05-11 07:13

from django.db import migrations, models


def fill_taskid(apps, schema_editor):
    # be sure to change it to your app here
    Task_Master_Data = apps.get_model("taskapp", "Task_Master_Data")
    db_alias = schema_editor.connection.alias
    tasks = Task_Master_Data.objects.using(db_alias).all()
    for i, task in enumerate(tasks):
        task.task_id = i
    Task_Master_Data.objects.using(db_alias).bulk_update(tasks, ["task_id"])


class Migration(migrations.Migration):
    dependencies = [
        ('taskapp', '0010_task_master_data'),
    ]

    operations = [
        migrations.AddField(
            model_name='task_master_data',
            name='task_id',
            field=models.IntegerField(null=True),
        ),
        migrations.RunPython(fill_taskid),
        migrations.RemoveField(
            model_name='task_master_data',
            name='id',
        ),
        migrations.AlterField(
            model_name='task_master_data',
            name='task_id',
            field=models.AutoField(default=-1, primary_key=True, serialize=False),
            preserve_default=False,
        ),
    ]

Note that depending on the state of your database you might need to rollback some of your migrations. The migration I created should work if your database is in a state where your task_id field does not exist yet.