I have a Django-Celery-PostgreSQL project that should partially run in Docker. Having Redis and PostgreSQL servers running locally on the machine and using virtual environment for the project everything runs smoothly. But when I try to setup Docker instance Celery seems unable to connect to the Postgres database while Django can.
When running project in Docker I put only Django and Celery in it and set network_mode: "host". Redis and Postgres remains on the local machine. Django server itself works flawlessly reading and writing data into the Postgres database, but when I try to run Celery tasks - I get following exception (thrown by django_celery_results):
[2019-08-13 11:26:24,815: ERROR/MainProcess] Pool callback raised exception: OperationalError('could not connect to server: No such file or directory\n\tIs the server running locally and accepting\n\tconnections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?\n',)
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
self.connect()
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 195, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 178, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/billiard/pool.py", line 1750, in safe_apply_callback
fun(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/celery/worker/request.py", line 371, in on_failure
store_result=self.store_errors,
File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 160, in mark_as_failure
traceback=traceback, request=request)
File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 342, in store_result
request=request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django_celery_results/backends/database.py", line 38, in _store_result
using=using,
File "/usr/local/lib/python3.6/site-packages/django_celery_results/managers.py", line 50, in _inner
return fun(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django_celery_results/managers.py", line 129, in store_result
defaults=fields)
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 538, in get_or_create
return self.get(**kwargs), False
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 402, in get
num = len(clone)
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __len__
self._fetch_all()
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1098, in execute_sql
cursor = self.connection.cursor()
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 256, in cursor
return self._cursor()
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 233, in _cursor
self.ensure_connection()
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
self.connect()
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
self.connect()
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 195, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 178, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Docker-compose:
version: '3'
services:
road_data_service:
build:
context: ./
dockerfile: ./_docker/backend/local_dev/Dockerfile
volumes:
- ./:/server
network_mode: "host"
ports:
- "8326:8000"
Dockerfile:
RUN apt-get update \
&& apt-get install -y binutils libproj-dev gdal-bin \
&& rm -rf /var/lib/apt/lists/*
WORKDIR ./
COPY ./ /server
COPY _docker/backend/local_dev/entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
WORKDIR /server/
CMD python3 manage.py runserver 0.0.0.0:8000 && celery -A worker_name \
worker -l info
