2

Long story short, something's going on with Django migrations and user models that I don't understand. Something about foreign key relations with a customer user auth model and migrations is borked.

I developed the app localy with SQLite, and a custom user object: class Client(AbstractBaseUser, PermissionsMixin) In my settings.py, I have:

AUTH_USER_MODEL = "offerdrive.Client"

Everything's worked with SQLite and local/development, and the app has a reasonable featureset at this point.

When time came to move to production, I tried Postgres, and it failed. Then I tried MySQL on RDS, and it failed the same way. Here's the MySql error after a ./manage.py migrate:

django.db.utils.IntegrityError: (1215, 'Cannot add foreign key constraint')

Here's some InnoDB debugging information for the curious (from mysql> show engine innodb status\G) -

------------------------
LATEST FOREIGN KEY ERROR
------------------------
2015-08-05 22:01:36 2af53b3cf700 Error in foreign key constraint of table test_drivedb/#sql-a69_248:
 FOREIGN KEY (`user_id`) REFERENCES `offerdrive_client` (`id`):
Cannot resolve table name close to:
 (`id`)

I "flattened" all of my migrations the "old-fashioned" way to be sure - rm -rf'ed the migrations directory, and redid makemigrations.

Some stuff I've read, and tried, which has failed to fully rectify the issue:

[1] Django 1.8 RC1: ProgrammingError when creating database tables.

[2] https://code.djangoproject.com/ticket/24524

Tests still fail the same way, unfortunately - even if the normal/prod database is migrated properly, and the migrations work consistently, the test database stuffs fail with the same foreign key issue.

I can get the actual database working fine if I'm careful about the order in which I migrate:

./manage.py reset_db
./manage.py migrate auth
./manage.py offerdrive
./manage migrate

But, this doesn't seem to hold for tests- it doesn't seem like running ./manage.py test offerdrive enforces the order in any way. It fails the same way - complaining about the same foreign key relations.

This seems like an absolutely silly issue for a framework like Django, and I'm thinking I'm doing something Really Silly. The app itself doesn't do anything weird or crazy. It's still baffling to me why it worked fine with SQLite but not Postgres or MySQL. My best guess is that SQLite is less strict with foreign key constraints somehow, and that even if the tables are created out of order, they will eventually resolve themselves. That, or something else. Possibly.

tl;dr computers are hard

Any thoughts?

Update - Apparently running migrate twice in a row will work for the 'real' database - but this doesn't help for tests, which fail.

Community
  • 1
  • 1
mallyvai
  • 1,698
  • 14
  • 20

2 Answers2

2

Hoo boy. So after some toil, some correspondence with other folk, and a bit of luck, the best solution turned out to be simple - I needed to pip install --upgrade django-registration-redux .

1) The turning point was (and I can't believe I didn't think of this sooner) enabling additional verbosity on the ./manage.py migrate -v3 and seeing where exactly it failed (before, I tried dropping debuggers through the actual django migration loading / execution code, which turned out to be the wrong level of abstraction to look at it with)

2) Looking at the verbose migration output, it was apparent things were bombing in something around the registration_profile tables. (it was the last thing running before the migration failed).

3) I dug around the package (django-registration / django-registration-redux) that app corresponded to... aaaand, it looked like there were no migrations. This is a problem because Be aware, however, that unmigrated apps cannot depend on migrated apps, by the very nature of not having migrations. according to ""https://docs.djangoproject.com/en/1.8/topics/migrations/#dependencies

4) On a whim, before trying to add my own migrations to the module, I upgraded django-registration-redux and django-registration with pip install --upgrade

Aaaand it fixed it. Hooray #engineering!

I want to note this had nothing to do with the custom user module, just an out-of-date dependency I didn't even know was entering the picture.

mallyvai
  • 1,698
  • 14
  • 20
0

After trying various methods to solve the problem I finally removed mysql from my system completely (if you have important databases then first make a backup) and reinstalled.

Then I repeated the usual steps :-

$./manage.py makemigrations
$./manage.py migrate

and it worked. Remember, migrate "auth" first and then other apps.

anirudhajain
  • 14
  • 1
  • 3