`table "django_migrations" already exists` error with Python 3 and SQLite3

Question

I've recently started receiving an error that I'm having trouble identifying with a Django project using SQLite. The project linked below was/is previously Python 2/3 compatible, though I primarily ran it using Python 2. Recently, I switched over most of my computers and projects to default to using Python 3 (I know, better late than never, right?). Additionally, I upgraded the project from Django 1.7 to 1.11. After this, the project started receiving the table already exists error, but only when running the migrate command using python3.

I also only get the error when running python3 manage.py migrate. For instance, python3 manage.py test works just fine, which is a bit baffling given test first runs the migrations. Running python2 manage.py migrate works just fine, no errors.

I am running Python 3.6.4, installed via Homebrew on a Mac, and the error I received is:

File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 59, in ensure_schema
   raise MigrationSchemaMissing("Unable to create the django_migrations table (%s)" % exc)
django.db.migrations.exceptions.MigrationSchemaMissing: Unable to create the django_migrations table (table "django_migrations" already exists)

I run into this exact same issue—broken with Python 3 but working with Python 2—on multiple computers, so I'd imagine anyone else will see the same issue. You should be able to reproduce the issue with the following steps:

1. git clone git@github.com:alexdlaird/django-bootstrap-authentication-template-project.git && cd django-bootstrap-authentication-template-project
2. make install
3. python2 manage.py migrate - note that it works just fine
4. rm db.sqlite for a fresh start
5. python3 manage.py migrate - note that it fails with the error shown above 5x. rm db.sqlite for a fresh start 5x. python3 manage.py test - note that it works just fine

To see the migrations run against a MySQL instance (the project assumes you have a default Homebrew MySQL instance running locally, but this can be configured in .env if not), run python3 manage.py migrate and observe that this works just fine with Python 2 or 3, so the issue appears isolated to SQLite migrations.

Any ideas? What am I doing wrong here?

-- .pyc Update --

I have executed find . -name \*.pyc -delete between running the above commands, but this does nothing to alleviate the situation. This makes sense, given the issue persists on a fresh clone, but I think we can eliminate it being an issue with Python's cached files.

Answer 1

Finally found it. For others that may run into a similar issue, here is the offending lines in my project that I simply had to remove: https://github.com/alexdlaird/django-bootstrap-authentication-template-project/commit/db16ff88d0d6c25eed38e52bd8332c721ed21e2f?diff=split#diff-e398a065684e871bec35f76ea80f20a7

This was an artifact of an ages old hack used for Python 2.5/6 and SQLite (see here)—it's by no means a good solution, but given SQLite was only used for dev and testing, it worked just fine. It was certainly better than banging my head against Python 2/SQLite unicode incompatibilities for hours on end.

Not surprisingly, I forgot about this hack long ago, and it didn't occur to me until I was debugging the Django code (describe in the comment above) and realized why the string tables names had ugly byte wrappers.

Answer 2

Clear your django migration cache before each try.

find . -path "*/migrations/*.pyc"  -delete