How to resolve the "psycopg2.errors.UndefinedTable: relation "auth_user" does not exist" when running django unittests on Travis

Question

I'm using Travis for CI/CD as part of my Django app, with a postgresql database. (Django 2.1.4)

The build consistently fails on Travis as soon as the tests run. I receive this error:

psycopg2.errors.UndefinedTable: relation "auth_user" does not exist

I have tried: makemigrations, migrate auth, migrate myapp, migrate --run-syncdb. All of which fail with the same error.

The tests run locally with a sqlite3 database, and on a prod-like heroku environment with a postgresql database.

.travis.yml

...
before script:
-psql -c 'create database travis_ci_test;' -U postgres
services:
-postgresql
script:
-yes | python3 manage.py makemigrations
-python3 manage.py migrate auth
-python3 manage.py migrate --run-syncdb
-python3 manage.py tests test/unit_tests

settings.py

...
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'travis_ci_test',
        'USER': 'postgres',
        'PASSWORD': '',
        'HOST': 'localhost',
        }
    }

...
INSTALLED_APPS = [...
        'django.contrib.auth',
        ]

Here is the output from the failing build on Travis. 'migrate auth' is successful (I think this is the crucial line for auth_user : Applying auth.0001_initial... OK)

0.22s$ psql -c 'create database travis_ci_test;' -U postgres
CREATE DATABASE
1.50s$ yes | python3 manage.py makemigrations
TEST_ENV...
AWS_INTEGRATION...
Databases ... {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'travis_ci_test', 'USER': 'postgres', 'PASSWORD': '', 'HOST': 'localhost'}
Installed Apps ... ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'races.apps.RacesConfig', 'storages']
No changes detected
The command "yes | python3 manage.py makemigrations" exited with 0.
1.68s$ python3 manage.py migrate auth
TEST_ENV...
AWS_INTEGRATION...
Databases ... {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'travis_ci_test', 'USER': 'postgres', 'PASSWORD': '', 'HOST': 'localhost'}
Installed Apps ... ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'races.apps.RacesConfig', 'storages']
Operations to perform:
  Apply all migrations: auth
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
The command "python3 manage.py migrate auth" exited with 0.
1.57s$ python3 manage.py migrate --run-syncdb
TEST_ENV...
AWS_INTEGRATION...
Databases ... {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'travis_ci_test', 'USER': 'postgres', 'PASSWORD': '', 'HOST': 'localhost'}
Installed Apps ... ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'myapp.apps.MyAppConfig', 'storages']
Operations to perform:
  Synchronize unmigrated apps: messages, myapp, staticfiles, storages
  Apply all migrations: admin, auth, contenttypes, sessions
Synchronizing apps without migrations:
  Creating tables...
    Creating table myapp_model1
    Creating table myapp_model2
    Creating table myapp_model3
    Creating table myapp_model4
    Creating table myapp_model5
    Running deferred SQL...
Running migrations:
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying sessions.0001_initial... OK
The command "python3 manage.py migrate --run-syncdb" exited with 0.
1.40s$ python3 manage.py test tests/unit_tests
TEST_ENV...
AWS_INTEGRATION...
Databases ... {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'travis_ci_test', 'USER': 'postgres', 'PASSWORD': '', 'HOST': 'localhost'}
Installed Apps ... ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'app.apps.MyAppConfig', 'storages']
Creating test database for alias 'default'...
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "auth_user" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/commands/test.py", line 26, in run_from_argv
    super().run_from_argv(argv)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/commands/test.py", line 56, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/test/runner.py", line 604, in run_tests
    old_config = self.setup_databases()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/test/runner.py", line 551, in setup_databases
    self.parallel, **kwargs
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/test/utils.py", line 174, in setup_databases
    serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE', True),
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/base/creation.py", line 68, in create_test_db
    run_syncdb=True,
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/__init__.py", line 148, in call_command
    return command.execute(*args, **defaults)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 172, in handle
    self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 310, in sync_apps
    self.stdout.write("    Running deferred SQL...\n")
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 106, in __exit__
    self.execute(sql)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 133, in execute
    cursor.execute(sql, params)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "auth_user" does not exist
The command "python3 manage.py test tests/unit_tests" exited with 1.

Probably not the problem, but you shouldn't be running makemigrations here. Makemigrations is something you do locally, after which you add the generated migration files to your version control. — Daniel Roseman, May 29 '19 at 08:02
Your database setup is irrelevant for running tests, since the Django test runner will create (and destroy) an entirely new database when it's running. So what you do prior to running `manage.py test` is not related to the tests failing. Read [this](https://docs.djangoproject.com/en/2.2/topics/testing/overview/#the-test-database). I don't know why your test is failing. Do you access your database on module level? — dirkgroten, May 29 '19 at 09:18
Something is off here: the `INSTALLED_APPS` you posted does not match the output `Installed Apps ... ['django.contrib.admin', ...` — Daniel Hepper, May 29 '19 at 09:21
@DanielHepper - I just used the ... to save showing all of the INSTALLED_APPS in settings.py - the list in the fail script matches those listed in settings.py — 741852963, May 29 '19 at 10:18
I see. Try running your tests with `--verbosity 3`, maybe that provides a useful hint. — Daniel Hepper, May 29 '19 at 11:26
@DanielRoseman - weirdly, that was key to solving it. Removing that step made it fail due to there being no migrations for the app. This made me realise that the migrations file was not getting sent to version control. One __init__.py file in the migrations folder later and it's working. — 741852963, May 31 '19 at 08:01

score 21 · Accepted Answer · edited Jan 22 '20 at 01:03

21

I ran into the same problem after adding a model for an app that was already installed, but forgetting to creating the initial migration. OP hid the answer in the comments so here are explicit steps:

$ mkdir myapp/migrations
$ touch myapp/migrations/__init__.py

After this I got a new error when trying to run python manage.py test

psycopg2.errors.InvalidCursorName: cursor "_django_curs_140073820227328_58" does not exist

One more step was needed:

$ python manage.py makemigrations

And then tests ran fine.

edited Jan 22 '20 at 01:03

Alexander Tyapkov

4,837
5
39
65

answered Dec 03 '19 at 18:47

Tom McCarty

326
3
3

2

This appears to happen with *any* inconsistency between your models and migration files when running unit tests. – Lord Elrond Jan 16 '20 at 22:44

score 4 · Answer 2 · answered May 06 '20 at 15:44

4

I deleted the sqllite3 database then I ran

$ python manage.py migrate

then

$ python manage.py migrate --run-syncdb

answered May 06 '20 at 15:44

Mohammed Nagdy

94
3

score 2 · Answer 3 · answered Jul 07 '21 at 22:37

2

The issue is that one of your apps is calling or referencing a non-existent model. The model may be in your code but no migration has been run for it.

Fix: Make sure all your apps have a migration package (i.e. migration folder with init.py file), then run makemigrations and migrate.

answered Jul 07 '21 at 22:37

Femolak

31
4

1

This fixed the issue for me. Somehow the app that creates a custom User Model didn't have any migrations folder created on initially running makemigrations, so I had to run it separately first for that app before I could migrate – Vasco Jul 06 '22 at 09:17

How to resolve the "psycopg2.errors.UndefinedTable: relation "auth_user" does not exist" when running django unittests on Travis

3 Answers3

Linked