0

Hej all,

I am trying to integrate a SQLite legacy database with Django v3.1.2, and so far it's pretty painful (new to django, so apologies if it's sth obvious). I assume something is wrong with my database schema or with the migration process. So here is what I did and my problem:

I have run inspectdb on my legacy database and cleaned up the models, leaving me with the following two models (there are more models in my application, but these are the ones apparently causing a problem):

class Integrons(models.Model):
    arg = models.ForeignKey('Args', on_delete=models.CASCADE)
    int_id = models.IntegerField(primary_key=True)
    int_start = models.IntegerField()
    int_stop = models.IntegerField()
    int_complete = models.CharField(max_length=500)

    class Meta:
        managed = False
        db_table = 'integrons'

class IntElements(models.Model):
    int = models.ForeignKey('Integrons', on_delete=models.CASCADE)
    el_id = models.IntegerField()
    el_start = models.IntegerField()
    el_stop = models.IntegerField()
    el_name = models.CharField(blank=True, null=True, max_length=500)
    el_strand = models.CharField(blank=True, null=True, max_length=500)

    class Meta:
        managed = False
        db_table = 'int_elements'

The respective fields in my legacy database are:

Integrons: arg_id, int_id, int_start, int_stop, int_complete IntElements: int_id, el_id, el_start, el-stop, el_name, el_strand

As seen in the models, IntElements.int_id should refer to Integrons.int_id.

Now I am trying to migrate everything - running python manage.py makemigrations works fine. However, when running python manage.py migrate , I get the following error:

django.db.migrations.exceptions.MigrationSchemaMissing: Unable to create the django_migrations table (foreign key mismatch - "int_elements" referencing "integrons")

I don't understand what the problem is. The first time I ran this I thought the problem was my model order, as I had defined IntElements before Integrons in models.py. I changed it, deleted the .pyc files in the migration folder and repeated the migration process, getting the same error. When I delete the migration files and comment out the IntElements model in models.py and run makemigrations, I see that all models except IntElements are created. But when I then run python manage.py migrate, I get the same error, which leads me to believe that something is going wrong with the migration process. Or could it be the _id in my column names?

Would really apprechiate any help on this!

EDIT_____

python manage.py showmigrations

outputs

    python manage.py showmigrations
System check identified some issues:

WARNINGS:
db_app.Lineages.taxon: (fields.W342) Setting unique=True on a ForeignKey has the same effect as using a OneToOneField.
    HINT: ForeignKey(unique=True) is usually better served by a OneToOneField.
admin
 [ ] 0001_initial
 [ ] 0002_logentry_remove_auto_add
 [ ] 0003_logentry_add_action_flag_choices
auth
 [ ] 0001_initial
 [ ] 0002_alter_permission_name_max_length
 [ ] 0003_alter_user_email_max_length
 [ ] 0004_alter_user_username_opts
 [ ] 0005_alter_user_last_login_null
 [ ] 0006_require_contenttypes_0002
 [ ] 0007_alter_validators_add_error_messages
 [ ] 0008_alter_user_username_max_length
 [ ] 0009_alter_user_last_name_max_length
 [ ] 0010_alter_group_name_max_length
 [ ] 0011_update_proxy_permissions
 [ ] 0012_alter_user_first_name_max_length
contenttypes
 [ ] 0001_initial
 [ ] 0002_remove_content_type_name
db_app
 [ ] 0001_initial
sessions
 [ ] 0001_initial
sequence_hard
  • 5,115
  • 10
  • 30
  • 50
  • Both are unmanaged models, why are they in migrations? Did you unmark some as managed and kept these as unmanaged? –  Oct 29 '20 at 13:32
  • I created the models using 'inspectdb', so I just left them to be unmanaged and run my 'makemigrations' and 'migrate' commands, I did not change anything in the meta models... – sequence_hard Oct 29 '20 at 14:02
  • This is hard to debug. I suspect there's a reference from a managed model to the unmanaged one and now it's complaining. Could you include the output of `showmigrations`? Also, `makemigrations` should not actually create anything if all you did is inspectdb + cleanup. –  Oct 29 '20 at 14:12
  • Hej, I added the showmigrations output to the question. There cannot really be a reference from a manages model to an unmanaged one, all my models are unmanaged... – sequence_hard Oct 29 '20 at 16:29

1 Answers1

1

Something is amiss here, because unmanaged models do not create migrations:

By default, inspectdb creates unmanaged models. That is, managed = False in the model’s Meta class tells Django not to manage each table’s creation, modification, and deletion

[code sample cut]

If you do want to allow Django to manage the table’s lifecycle, you’ll need to change the managed option above to True (or remove it because True is its default value).

This is intentional as the most prominent case of legacy databases are read-only databases that provide information that is being phased out. Using Django to manage them would disrupt the legacy application and may not even be possible if proper permissions are set for schema operations.

You also mention tables are being created which again conflicts with unmanaged tables.

If your goal is to import the schema, then delete the SQLite database and have it properly populated and managed by Django, then you must make the models managed by deleting managed = False from the model Meta class. This is the other common use case in which you inherit a database and want to start from that and move forward using the migration framework. Inspectdb is a one-off command then to have your boilerplate in place.

  • Thanks for your answer! The goal is to import a read only database and use django with that. It makes sense that the migrations should not output anything for unmanaged models, but they clearly do in this case... – sequence_hard Oct 29 '20 at 16:32
  • One last test: if you remove "db_app" from `INSTALLED_APPS` in your settings, then run migrate, it should not give any problem and the standard django tables should be created. There's something wrong in code you're not showing, but I can't tell you where to look, cause I don't understand how you got unmanaged models to generate migrations. If all models in db_app have managed = False, there should not be a 0001_initial there. –  Oct 29 '20 at 17:21
  • I spent the whole day on this problem and managed to fix it nearly completely. If I do not run makemigrations and migrate, I can just use my models in the shell and it works (nearly) fine. the main problem here was/is how I named the fields in my models, lots of them had the suffix _id, which led to clashes as django appends and _id to the fields in some instances. I have one model left to fix, will report back once i have managed that. – sequence_hard Oct 29 '20 at 19:23
  • So then either inspectdb messed up or your clean up removed those suffixes, generating missing foreign keys. It still is unexpected that migrations are created. I was going to get to "int" being used as field (which is a builtin function in python), but I didn't see that to be related to this. –  Oct 29 '20 at 19:39
  • 1
    Managed to fix everything now, and the issue was indeed the naming. will keep that in mind when using inspectdb the next time. Thanks for your help, really appreciate it! – sequence_hard Oct 30 '20 at 08:43