I have an AWX deployment using docker compose. The postgres database is having issues with dns name resolution. The issue started after I updated the docker-compose.yml to force the awx_web container to write some DNS IPs in /etc/resolv.conf. After that I ran:
docker pull; docker compose up -d
The containers were recreated, and I could no longer connect to the AWX web interface. I have tried stopping the containers, restarting them, reloading nginx and confirm it's running, recreating the containers with tweaked docker-compose.yml's, and I keep running into the server error issue.
$ awx logs awx_web
self.connect()
File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/base/base.py", line 195, in connect
self.connection = self.get_new_connection(conn_params)
File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/postgresql/base.py", line 178, in get_new_connection
connection = Database.connect(**conn_params)
File "/var/lib/awx/venv/awx/lib/python3.6/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: could not translate host name "postgres" to address: Name or service not known
RESULT 2
OKREADY
2023-03-15 02:16:01,844 INFO exited: wsbroadcast (exit status 1; not expected)
2023-03-15 02:16:01,844 INFO exited: wsbroadcast (exit status 1; not expected)
2023-03-15 02:16:02,848 INFO spawned: 'wsbroadcast' with pid 7561
2023-03-15 02:16:02,848 INFO spawned: 'wsbroadcast' with pid 7561
2023-03-15 02:16:03,850 INFO success: wsbroadcast entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-03-15 02:16:03,850 INFO success: wsbroadcast entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
post gres logs
PostgreSQL Database directory appears to contain a database; Skipping initialization
2023-03-15 02:14:16.396 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-03-15 02:14:16.396 UTC [1] LOG: listening on IPv6 address "::", port 5432
2023-03-15 02:14:16.404 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-03-15 02:14:16.432 UTC [26] LOG: database system was shut down at 2023-03-15 02:13:50 UTC
2023-03-15 02:14:16.440 UTC [1] LOG: database system is ready to accept connections
NETWORK ID NAME DRIVER SCOPE
5c661c37460f awxcompose_default bridge local
6e91f0243cfd bridge bridge local
0f6ab0c7ce0f host host local
5c7e12dba410 none null local
Going to the AWX home page shows this server error.
My docker-compose.yml
services:
web:
image: ansible/awx:15.0.0
container_name: awx_web
depends_on:
- redis
- postgres
ports:
- "80:8052"
- "443:443"
hostname: awxweb
user: root
restart: unless-stopped
volumes:
- supervisor-socket:/var/run/supervisor
- rsyslog-socket:/var/run/awx-rsyslog/
- rsyslog-config:/var/lib/awx/rsyslog/
- "~/.awx/awxcompose/SECRET_KEY:/etc/tower/SECRET_KEY"
- "~/.awx/awxcompose/environment.sh:/etc/tower/conf.d/environment.sh"
- "~/.awx/awxcompose/credentials.py:/etc/tower/conf.d/credentials.py"
- "~/.awx/awxcompose/nginx.conf:/etc/nginx/nginx.conf:ro"
- "~/.awx/awxcompose/redis_socket:/var/run/redis/:rw"
- "/var/lib/awx/projects:/var/lib/awx/projects:rw"
- "./resolv.conf:/etc/resolv.conf"
dns:
- 127.0.0.11
environment:
http_proxy:
https_proxy:
no_proxy:
task:
image: ansible/awx:15.0.0
container_name: awx_task
depends_on:
- redis
- web
- postgres
command: /usr/bin/launch_awx_task.sh
hostname: awx
user: root
restart: unless-stopped
volumes:
- supervisor-socket:/var/run/supervisor
- rsyslog-socket:/var/run/awx-rsyslog/
- rsyslog-config:/var/lib/awx/rsyslog/
- "~/.awx/awxcompose/SECRET_KEY:/etc/tower/SECRET_KEY"
- "~/.awx/awxcompose/environment.sh:/etc/tower/conf.d/environment.sh"
- "~/.awx/awxcompose/credentials.py:/etc/tower/conf.d/credentials.py"
- "~/.awx/awxcompose/redis_socket:/var/run/redis/:rw"
- "/var/lib/awx/projects:/var/lib/awx/projects:rw"
dns:
#- 10.1.2.3
#- 10.2.3.4
- 127.0.0.11
environment:
http_proxy:
https_proxy:
no_proxy:
SUPERVISOR_WEB_CONFIG_PATH: '/etc/supervisord.conf'
redis:
image: redis
container_name: awx_redis
restart: unless-stopped
environment:
http_proxy:
https_proxy:
no_proxy:
command: ["/usr/local/etc/redis/redis.conf"]
volumes:
- "~/.awx/awxcompose/redis.conf:/usr/local/etc/redis/redis.conf:ro"
- "~/.awx/awxcompose/redis_socket:/var/run/redis/:rw"
postgres:
image: postgres:10
container_name: awx_postgres
restart: unless-stopped
volumes:
- "~/.awx/pgdocker/10/data/:/var/lib/postgresql/data:Z"
environment:
POSTGRES_USER: awx
POSTGRES_PASSWORD: awxpass
POSTGRES_DB: awx
http_proxy:
https_proxy:
no_proxy:
volumes:
supervisor-socket:
rsyslog-socket:
rsyslog-config:
I also tried redploying the environment using the awx/installer/install.yml, but the issue persists.