Add Django model parent class to an existing "child" model for multi-table inheritance

Question

I would like to add a new parent class to an existing model, which inherits from the same "super" parent class. Example of initial condition:

from django.db import models

class Ticket(PolymorphicModel):
    name = models.CharField(max_length=50)
    company = models.CharField(max_length=80)
    price = models.CharField(max_length=10)

class MovieTicket(Ticket):
    # functions and logic

For my implementation, I would like to add an "intermediary" EntertainmentTicket model, which inherits from Ticket and is inherited by MovieTicket for logic grouping purposes. Desired final condition:

class Ticket(PolymorphicModel):
    name = models.CharField(max_length=50)
    address = models.CharField(max_length=80)
    price = models.CharField(max_length=10)

class EntertainmentTicket(Ticket):
    # some functions and common logic extracted

class MovieTicket(EntertainmentTicket):
    # logic unique to MovieTicket

Note that the child classes have the same fields as Ticket, they only contain functions and logic. I cannot make Ticket into abstract, because I have other models pointing to it in a foreign key relationship. I use django-polymorphic to return the appropriate ticket type.

I have made migrations for EntertainmentTicket, I presume the next step is to create instances of EntertainmentTicket that point to the same Ticket instances that the current MovieTickets are pointing to. What is the best way to do this?

Nealium · Accepted Answer · 2023-03-28T01:52:56.013

As long as you aren't adding extra fields to these two models, you can make them into proxy ones.

class EntertainmentTicket(Ticket):
    class Meta:
      proxy = True

    # some functions and common logic extracted

class MovieTicket(EntertainmentTicket):
    class Meta:
        proxy = True

    # logic unique to MovieTicket

This has no effect on the database and it's just how you interact with them in Django itself

Edit

I completely missed that PolymorphicModel and wasn't aware of it actually making database tables..
Proxy model's wouldn't really fit your existing scheme, so this is what I would do:

1. Rename `MovieTicket` to `EntertainmentTicket`

2. Create a new Model `MovieTicket`

This could be done in two migrations, but I did makemigrations after each step and then copied them into 1 & deleted #2

Example End Migration:

Order Matters!!

# Generated by Django 3.2.4 on 2023-03-27 15:29

from django.db import migrations, models
import django.db.models.deletion

class Migration(migrations.Migration):

    dependencies = [
        ('contenttypes', '0002_remove_content_type_name'),
        ('child', '0001_initial'),
    ]

    operations = [
        migrations.RenameModel(
            old_name='MovieTicket',
            new_name='EntertainmentTicket',
        ),
        migrations.CreateModel(
            name='MovieTicket',
            fields=[
                ('entertainmentticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.entertainmentticket')),
            ],
            options={
                'abstract': False,
                'base_manager_name': 'objects',
            },
            bases=('child.entertainmentticket',),
        ),
    ]

3. Handle old `MovieTicket` objects (now `EntertainmentTicket`)

# Create Movie Ticket objects pointing to the new model
for i in EntertainmentTicket.objects.all():
  MovieTicket.objects.create(entertainmentticket_ptr_id=i.pk)

I did all this in a test project and it all worked, plus it keeps your PolymorphicModel thing

Edit 2

ya just had to make it more complicated, didn't you?!- this was horrible! but i did learn alot.

I recommend making a temp project and playing around with it before trying anything in Production.

Starting Models:

from django.db import models
from polymorphic.models import PolymorphicModel

class Ticket(PolymorphicModel):
    name = models.CharField(max_length=50)
    company = models.CharField(max_length=80)
    price = models.CharField(max_length=10)

class MovieTicket(Ticket):
    pass

class MarvelTicket(MovieTicket):
    pass

class TheatreTicket(Ticket):
    pass

Starting Data:

# Note my app was named child
from child.models import *
MoveTicket.objects.create(name='test movie', company='test company', price='$5')
MarvelTicket.objects.create(name='Ant Man', company='Marvel', price='$250')
TheatreTicket.objects.create(name='Singing in the Rain', company='Gene Kelly ', price='$2')

Updated Models:

from django.db import models
from polymorphic.models import PolymorphicModel

class Ticket(PolymorphicModel):
    name = models.CharField(max_length=50)
    company = models.CharField(max_length=80)
    price = models.CharField(max_length=10)

class EntertainmentTicket(Ticket):
    pass

class MovieTicket(EntertainmentTicket):
    pass

class MarvelTicket(MovieTicket):
    pass

class TheatreTicket(EntertainmentTicket):
    pass

Migrations

See: Moving a Django Foreign Key to another model while preserving data? for the blueprint of how

You'll need to change the child in all of the apps.get_model('child', 'Ticket') calls to match your app

Migration #1. Temp Model, Deleting.

Steps:

Create Temp Model
Pack Temp Model
Remove all downstream affected Models
- One-to-One Primary Keys screw everything up.
- We must delete marvel because we must delete movie
Create new 'top' / Entertainment Model
Create Entertainment objects

# Generated by Django 3.2.4 on 2023-03-27 23:17

from django.db import migrations, models
import django.db.models.deletion

def create_transfer_objects(apps, schema_editor):
    """
    Pack Movie, Theatre and Marvel into the temp model

    Structure of temp:
        pk          = auto-generated number
        ticket_type = two char string
        ticket_pk   = PK for parent Ticket

    Explications:
        pk, needs a pk
        ticket_type, so it knows what model to create with later

        ticket_pk, what parent ticket it's associated with

            Note Marvel still stores parent ticket and fetch Movie from ticket
                I just found this easier instead of adding more temp fields
    """

    transfer_model = apps.get_model('child', 'TransferModel')

    # create direct -> ticket items
    entertainment_models = [
        ['00', apps.get_model('child', 'MovieTicket')],
        ['01', apps.get_model('child', 'TheatreTicket')],
    ]
    for t, m in entertainment_models:
        for m_obj in m.objects.all():
            transfer_model.objects.create(
                ticket_type=t,
                ticket_pk=m_obj.ticket_ptr.pk,
            )

    # create passthrough ticket items
    # pass through item's pk is still a ticket, just another name
    entertainment_models = [
        ['02', apps.get_model('child', 'MarvelTicket')],
    ]
    for t, m in entertainment_models:
        for m_obj in m.objects.all():
            if '02':
                # use movie ticket
                ticket_pk = m_obj.movieticket_ptr.pk
            # elif ...
            transfer_model.objects.create(
                ticket_type=t,
                ticket_pk=ticket_pk,
            )

def reverse_transfer_objects(apps, schema_editor):
    """
    Reverse the process of creating the transfer object,
    This will actually create the Movie, Theatre and Marvel objects

    """

    transfer_model = apps.get_model('child', 'TransferModel')
    ticket_model = apps.get_model('child', 'Ticket')

    # reverse direct -> ticket items
    target_dict = {
        '00': apps.get_model('child', 'MovieTicket'),
        '01': apps.get_model('child', 'TheatreTicket'),
    }
    for obj in transfer_model.objects.filter(ticket_type__in=target_dict.keys()):
        target_dict[obj.ticket_type].objects.create(
            ticket_ptr=ticket.objects.get(pk=obj.ticket_pk),
        )

    # reverse passthrough ticket items
    # Note: This only does "1 level" below
    target_dict = {
        '02': {
            'obj': apps.get_model('child', 'MarvelTicket'),
            'target': apps.get_model('child', 'MovieTicket'),
            'field': 'movieticket_ptr',
        },
    }
    for obj in transfer_model.objects.filter(ticket_type__in=target_dict.keys()):
        target_dict[obj.ticket_type]['obj'].objects.create(**{
            target_dict[obj.ticket_type]['field']: target_dict[obj.ticket_type]['target'].objects.get(pk=obj.ticket_pk),
        })

def migrate_to_entertainment(apps, schema_editor):
    """
    Create Entertainment objects from tickets
    """
    ticket_model = apps.get_model('child', 'Ticket')
    entertainmentticket_model = apps.get_model('child', 'EntertainmentTicket')
    for ticket_obj in ticket_model.objects.all():
        entertainmentticket_model.objects.create(
            ticket_ptr=ticket_obj
        )

def reverse_entertainment(apps, schema_editor):
    # Removing Model should do the trick
    pass

class Migration(migrations.Migration):

    dependencies = [
        ('child', '0001_initial'),
    ]

    operations = [
        # create a temp model to old values
        migrations.CreateModel(
            name='TransferModel',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('ticket_type', models.CharField(choices=[('00', 'Movie'), ('01', 'Theatre'), ('02', 'Marvel')], default='00', max_length=2)),
                ('ticket_pk', models.PositiveIntegerField(default=0)),
            ],
        ),

        # Create & Pack TransferModels
        migrations.RunPython(
            # This is called in the forward migration
            create_transfer_objects,
            # This is called in the backward migration
            reverse_code=reverse_transfer_objects
        ),

        # Delete
        migrations.DeleteModel(
            name='MovieTicket',
        ),
        migrations.DeleteModel(
            name='TheatreTicket',
        ),
        migrations.DeleteModel(
            name='MarvelTicket',
        ),

        # Create New Model
        migrations.CreateModel(
            name='EntertainmentTicket',
            fields=[
                ('ticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.ticket')),
            ],
            options={
                'abstract': False,
                'base_manager_name': 'objects',
            },
            bases=('child.ticket',),
        ),

        # Connect Entertainment to Ticket
        migrations.RunPython(
            # This is called in the forward migration
            migrate_to_entertainment,
            # This is called in the backward migration
            reverse_code=reverse_entertainment
        ),
    ]

Migration #2. Reforming

Steps:

Recreate deleted Models
Generate Objects using Temp Model, pointing at new upstream
Delete Temp Model

# Generated by Django 3.2.4 on 2023-03-28 01:01

from django.db import migrations, models
import django.db.models.deletion

def connect_to_entertainment(apps, schema_editor):
    """
    Recreate all the lost models using Temp Model,
    Connect through Entertainment instead of Ticket directly.
    """
    transfer_model = apps.get_model('child', 'TransferModel')
    entertainmentticket_model = apps.get_model('child', 'EntertainmentTicket')

    target_dict = {
        '00': apps.get_model('child', 'MovieTicket'),
        '01': apps.get_model('child', 'TheatreTicket'),
    }
    for obj in transfer_model.objects.filter(ticket_type__in=target_dict.keys()):
        target = entertainmentticket_model.objects.get(
            ticket_ptr__pk=obj.ticket_pk,
        )
        target_dict[obj.ticket_type].objects.create(
            entertainmentticket_ptr=target,
        )

    # Create passthrough ticket items
    target_dict = {
        '02': {
            'obj': apps.get_model('child', 'MarvelTicket'),
            'target': apps.get_model('child', 'MovieTicket'),
            'field': 'movieticket_ptr',
        },
    }
    for obj in transfer_model.objects.filter(ticket_type__in=target_dict.keys()):
        target_dict[obj.ticket_type]['obj'].objects.create(**{
            target_dict[obj.ticket_type]['field']: target_dict[obj.ticket_type]['target'].objects.get(
                pk=obj.ticket_pk
            ),
        })

def reverse_entertainment_connect(apps, schema_editor):
    """
    Recreate Temp Model from Movie, Theatre and Marvel.
    """
    transfer_model = apps.get_model('child', 'TransferModel')

    entertainment_models = [
        ['00', apps.get_model('child', 'MovieTicket')],
        ['01', apps.get_model('child', 'TheatreTicket')],
    ]
    for t, m in entertainment_models:
        for m_obj in m.objects.all():
            transfer_model.objects.create(
                ticket_type=t,
                ticket_obj=m_obj.entertainmentticket_ptr.ticket_ptr,
            )

    # Create passthrough ticket items
    entertainment_models = [
        ['02', apps.get_model('child', 'MarvelTicket')],
    ]
    for t, m in entertainment_models:
        for m_obj in m.objects.all():
            if '02':
                # use movie ticket
                ticket_pk = m_obj.movieticket_ptr.pk
            # elif ...
            transfer_model.objects.create(
                ticket_type=t,
                ticket_pk=ticket_pk,
            )

class Migration(migrations.Migration):

    dependencies = [
        ('child', '0002_transfer'),
    ]

    operations = [
        # Create the Previously Removed Models
        migrations.CreateModel(
            name='MovieTicket',
            fields=[
                ('entertainmentticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.entertainmentticket')),
            ],
            options={
                'abstract': False,
                'base_manager_name': 'objects',
            },
            bases=('child.entertainmentticket',),
        ),
        migrations.CreateModel(
            name='TheatreTicket',
            fields=[
                ('entertainmentticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.entertainmentticket')),
            ],
            options={
                'abstract': False,
                'base_manager_name': 'objects',
            },
            bases=('child.entertainmentticket',),
        ),
        migrations.CreateModel(
            name='MarvelTicket',
            fields=[
                ('movieticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.movieticket')),
            ],
            options={
                'abstract': False,
                'base_manager_name': 'objects',
            },
            bases=('child.movieticket',),
        ),

        #  Recreate
        migrations.RunPython(
            # This is called in the forward migration
            connect_to_entertainment,
            # This is called in the backward migration
            reverse_code=reverse_entertainment_connect
        ),

        # Delete Temp Model
        migrations.DeleteModel(
            name='TransferModel',
        ),

    ]

Run `python manage migrate`

Side Notes:

I know this is a drastic change and a lot of stuff, but holy it gets complicated quick!

I tried for the longest time to keep the original models and just repoint or rename, but because they were One-to-One and Primary Keys it was doomed from the start.

You can't really redo primary keys, or at least I couldn't find a way.
As long as that One-to-One existed, I couldn't create the EntertainmentTicket connection.

I also tried doing it in a single migration, but Django didn't like the immediate recreation of exact models. it was almost as if they weren't forgotten yet.

If you had another middle like model like:

class Ticket(PolymorphicModel):
    pass

class SpeakerTicket(Ticket):  # <- like this
    pass

class MovieTicket(Ticket):
    pass

# etc..

All you'd have to do is filter out those items when creating the EntertainmentTicket objects in the migrations

# Migrations #1
def migrate_to_entertainment(apps, schema_editor):
    """
    Create Entertainment objects from tickets
    """
    ticket_model = apps.get_model('child', 'Ticket')
    entertainmentticket_model = apps.get_model('child', 'EntertainmentTicket')


    ticket_object_list = ticket_model.objects.all()

    speaker_model = apps.get_model('child', 'SpeakerTicket')
    ticket_object_list = ticket_object_list.exclude(
        pk__in=speaker_model.objects.all().values_list('ticket_ptr')
    )

    # .. exclude another, etc

    for ticket_obj in ticket_object_list:
        entertainmentticket_model.objects.create(
            ticket_ptr=ticket_obj
        )

Thanks for the suggestion. I will test to see whether it works well with django-polymorphic. However suppose I went ahead and created the tables and already have MovieTicket data, what would be the right steps to correctly "insert" the EntertainmentTicket class? Something like: 1) Create an EntertainmentTicket for each Ticket that is a MovieTicket instance 2) change MovieTicket to inherit from EntertainmentTicket 3) update each MovieTicket instance's ptr to the corresponding EntertainmentTicket instance? — jenesaisquoi, Mar 27 '23 at 01:09
I've updated my answer. I was unaware of how PolymorphicModel's worked, they actually create db tables for each model.. so proxy models wouldn't really fit. — Nealium, Mar 27 '23 at 16:13
Thank you for the edit, I haven't thought of just renaming MovieTicket to EntertainmentTicket, that's brilliant. I researched a bit, and django-polymorphic docs seem to mention support for proxy models, but not a lot of details are given on how it is implemented — jenesaisquoi, Mar 27 '23 at 16:27
If I have another model with a foreign key pointing to the current MovieTicket instances, then I will need to update those as well correct? Also a follow up question: If I have another existing model TheatreTicket that also needs to extend from EntertainmentTicket, what would be the correct way to create the necessary EntertainmentTicket instances? — jenesaisquoi, Mar 27 '23 at 16:42
With more models it gets very, very complicated. You can't really **re-point** those primary keys, what you're basically going to have to do is: store the data, delete the effected models, recreate with new structure, fill tables. I've edited my answer and instead of renaming I leveraged `RunPython` in the migrations. You could also use a CSV dump and a Management Command instead of doing it all in the migrations, that's usually what I do but it's alot of moving parts and harder to explain; easy to do tho once you have the parts (imo) — Nealium, Mar 28 '23 at 02:02