0

On our staging server we observed a runtime error stating a field is missing from a database,

column our_table.our_field does not exist
LINE 1: ...d"."type", "our_table"...

The field was added during a recent update with a complicated migration squashing process. It's possible that some errors were made during this process, but "manage.py showmigrations" command shows that the migration has been applied and "manage.py makemigrations" does not create any new migration. As we do not run tests on our staging or production databases, we are trying to figure out the most effective method for identifying such errors.

In short, how can we identify mismatches between the database and Django models caused by an incorrect migration like the following?

python manage.py migrate our_app --fake

I suppose I am looking for something like

python manage.py check_database

Edit: Many thanks for the suggestions. However, this is more of a deployment than a development question because the problem likely occurred when our devops tried to apply the squashed migrations while retaining data on the staging server (which will be case on production). It was scary to learn that such inconsistency can occur when makemigrations and showmigrations do not show any problem and can therefore also happen on production.

The bottom line is that we need some way to ensure our database matches our models after deployment.

user2283347
  • 670
  • 8
  • 12
  • I don't know if there is a command for that in Django. I suppose you may have to write a module to inspect your db schema and compare it with your models. – kamran890 Feb 12 '23 at 15:35
  • The existing issue you can solve in number of ways ( go back to one revision, apply migraion again etc)and in future make sure no conflicts occurs. run makemigration in developer system ( and make migration in local), commit the migration file, NB: Do not run makemigration in staging or prod, In staging/prod only run migrate. Check the user 'SuperNov's answer in https://stackoverflow.com/questions/28035119/should-i-be-adding-the-django-migration-files-in-the-gitignore-file. – Jisson Feb 12 '23 at 16:06
  • If you want to follow this, you might need to fix the migration issues manually once in all developer systems ( by using fake, dump....), commit the migrations files from prod to repo ( once your staging == prod) , then all developers need to fetch the changes and fix the migration issues if any. – Jisson Feb 12 '23 at 16:20
  • @kamran890 This is possible but I am not how to do that without manually adding fields to the testing command when we execute something like `MyModel.objects.raw('SELECT field1, field2 FROM myapp_mymodel LIMIT 1')`. – user2283347 Feb 12 '23 at 23:19
  • to fix the issue of migration conflicts during tests, the following PYPI project is useful,https://pypi.org/project/django-test-without-migrations/, after installation, add it to installed app, then run python manage.py test --nomigrations – Jisson Feb 13 '23 at 04:57

1 Answers1

0

I posted the question to the django-extensions issue tracker and was told that the sqldiff command provided by django-extensions is the command that I am looking for. However, this command has a number of bugs, especially on the handling of constraints, so we cannot yet count on this tool for validating out databases.

user2283347
  • 670
  • 8
  • 12