0

The Error

I'm having a lot of trouble getting my Django CI/CD working. I am using Django + postgres, my test cases all pass but I want to implement continuous integration. Note the the following output is from the Gitlab Runner.

I keep getting this error: (the ... is where I've left out some detail, but its just traceback or name specific stuff about my project etc.)

$ cd src
$ python manage.py makemigrations
/usr/local/lib/python3.9/site-packages/django/core/management/commands/makemigrations.py:105: RuntimeWarning: Got an error checking a consistent migration history performed for database connection 'default': could not connect to server: Connection refused
    Is the server running on host "localhost" (::1) and accepting
    TCP/IP connections on port 5432?
could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5432?
  warnings.warn(
Migrations for 'app':
  app/migrations/0001_initial.py
    - Create model ...
...
...
$ python manage.py migrate
Traceback (most recent call last):
...
...
psycopg2.OperationalError: could not connect to server: Connection refused
    Is the server running on host "localhost" (::1) and accepting
    TCP/IP connections on port 5432?
could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5432?

Config files

.gitlab-cl.yml

image: python:latest

services:
  - postgres:latest
variables:
  POSTGRES_DB: postgres
  POSTGRES_HOST: postgres
  POSTGRES_USER: postgres
  POSTGRES_PASSWORD: postgres

cache:
  paths:
    - ~/.cache/pip/

before_script:
  - python -V  # Print out python version for debugging
  - pip install -r requirements.txt

test:
  script:
    - python manage.py makemigrations
    - python manage.py migrate
    - python manage.py test --settings mysite.cicd_settings

settings.py

As you can see, I have specified a different settings file cicd_settings to run my tests:

from mysite.settings import *

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'postgres',
        'USER': 'postgres',
        'PASSWORD': 'postgres',
        'HOST': 'postgres',
        'PORT': '5432',
   },
}

It inherits everything from my normal settings.py file, but overrides the database settings.

Tried Solutions

I have tried the following solutions:

Postgres Gitlab tutorial: https://docs.gitlab.com/ee/ci/services/postgres.html

Similar Stackoverflow question: GitLab CI Django and Postgres

Using dj_database_url package: Could not connect to server: Connection refused (0x0000274D/10061) - PostgreSQL on remote server

Default setup for Django Gitlab CI/CD https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Django.gitlab-ci.yml

As you can see, I am using a very simple setup for the tests, but I still can't get it working.

Heinrich
  • 340
  • 1
  • 4
  • 12

1 Answers1

1

The script fails at the python manage.py makemigrations line, since you don't specify the --settings for it. But actually, you don't even need to run migrations manually for tests - it will be automatically done by the test runner, to the test database.

So remove both python manage.py makemigrations and python manage.py migrate from the test block and it should run successfully.

Moreover, you would never want to run makemigrations automatically. It's a tool to be used in development, and the resulting migration are to be included in the VCS (git).

Alexandr Tatarinov
  • 3,946
  • 1
  • 15
  • 30
  • removing the 2 migration lines does break, but adding the `--setting` part to them does fix, so I suggest that you edit your answer to reflect this. I have submitted a guide to the Community Wiki here https://stackoverflow.com/questions/68243461/django-postgres-ci-cd-testing-guide – Heinrich Jul 04 '21 at 09:56
  • That sounds wrong. How exactly does it fail without that lines? Again, you should **never** have `makemigrations` in your CI code. You **only** run it manually on your local machine to produce migration files and commit them to VCS. – Alexandr Tatarinov Jul 04 '21 at 10:56
  • I've made a pastebin link for the output along with the exact yml file: https://pastebin.com/754htAB1 – Heinrich Jul 04 '21 at 12:39
  • Did you include **ALL** migrations files in your repository after running `makemigrations` locally? It looks like they are not there, hence the error – Alexandr Tatarinov Jul 04 '21 at 13:21
  • I don't follow? I don't commit the contents of my `migrations/` folder (other than `__init__.py`). Isn't it bad practice to commit migrations? Especially during development where the database is often empty and definitely during testing where an empty database is initialized. – Heinrich Jul 04 '21 at 15:16
  • No, you must commit migrations to the git. Check out the first page of the docs https://docs.djangoproject.com/en/3.2/topics/migrations/ or this question https://stackoverflow.com/questions/28035119/should-i-be-adding-the-django-migration-files-in-the-gitignore-file – Alexandr Tatarinov Jul 04 '21 at 19:13