Hello I came from this post Configure Nginx reverse proxy for MQTT
I have a virtual machine, in this virtual machine I have nginx proxy listening to the port 1885 where the clients connect, and I want to make it redistribute the messages among all the Mosquitto containers (In this example I delete the other two containers I have) this to create a mosquitto volume
version: '3'
services:
nginx:
image: nginx:latest
ports:
- "1885:1885"
- "443:443"
volumes:
- ./nginx.conf:/var/lib/docker/volumes/nginx-conf/_data/nginx.conf
networks:
- backend
mosquitto1:
image: eclipse-mosquitto
ports:
- "1886:1886"
environment:
- listener=1886:1886
- allow_anonymous=true
networks:
- backend
volumes:
- ./mosquitto.conf:/etc/mosquitto/mosquitto.conf
networks:
backend:
driver: bridge
Then I have the nginx.conf
events {
}
http {
upstream mosquitto_backend {
zone tcp_mem 64k;
proxy_protocol on;
}
server {
listen 1885;
server_name localhost;
log_format mqtt '$remote_addr [$time_local] $protocol $status $bytes_sent $bytes_received $session_time "$request" "$http_referer" "$http_user_agent" "$upstream_addr"';
access_log /var/log/nginx/mqtt_access.log mqtt;
error_log /var/log/nginx/mqtt_error.log; # Health check notifications
location / {
proxy_pass http://mosquitto_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
}
the problem I have is despite being able to suscribe to the port 1885 I cannot write messages from other terminal or connect using mqtt explorer
running docker-compose up will return this where it tries to use port 1883 and clashes with something
docker-compose up --remove-orphans
[+] Running 2/2
⠿ Container data-mosquitto1-1 Created 0.1s
⠿ Container data-nginx-1 Created 0.0s
Attaching to data-mosquitto1-1, data-nginx-1
data-mosquitto1-1 | 1678977906: mosquitto version 2.0.15 starting
data-mosquitto1-1 | 1678977906: Config loaded from /mosquitto/config/mosquitto.conf.
data-mosquitto1-1 | 1678977906: Starting in local only mode. Connections will only be possible from clients running on this machine.
data-mosquitto1-1 | 1678977906: Create a configuration file which defines a listener to allow remote access.
data-mosquitto1-1 | 1678977906: For more details see https://mosquitto.org/documentation/authentication-methods/
data-mosquitto1-1 | 1678977906: Opening ipv4 listen socket on port 1883.
data-mosquitto1-1 | 1678977906: Opening ipv6 listen socket on port 1883.
data-mosquitto1-1 | 1678977906: Error: Address not available
data-mosquitto1-1 | 1678977906: mosquitto version 2.0.15 running
data-nginx-1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
data-nginx-1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
data-nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
data-nginx-1 | 10-listen-on-ipv6-by-default.sh: info: IPv6 listen already enabled
data-nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
data-nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
data-nginx-1 | /docker-entrypoint.sh: Configuration complete; ready for start up
data-nginx-1 | 2023/03/16 14:45:06 [notice] 1#1: using the "epoll" event method
data-nginx-1 | 2023/03/16 14:45:06 [notice] 1#1: nginx/1.23.3
data-nginx-1 | 2023/03/16 14:45:06 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
data-nginx-1 | 2023/03/16 14:45:06 [notice] 1#1: OS: Linux 3.10.0-1160.83.1.el7.x86_64
data-nginx-1 | 2023/03/16 14:45:06 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
data-nginx-1 | 2023/03/16 14:45:06 [notice] 1#1: start worker processes
data-nginx-1 | 2023/03/16 14:45:06 [notice] 1#1: start worker process 22
I used lsof -i:1883 and netstat -tuln to see if this port is being used but is not being used
I know I'm using an image eclipse-mosquitto and this image shall have the port 1883 and despite changing later it access to it and no problem if that's the case, but what I don't know what else is accessing to it making the conflict.
Could you help me please?
EDIT
I made some changes and i'm able to suscribe to the port 1885 and in other terminal i can publish messages in the same port but the user suscribed can only see half of the messages
this is my docker-compose.yml
version: '3'
services:
nginx:
image: nginx:latest
container_name: nginx
ports:
- "1885:1885"
- "443:443"
volumes:
- /var/lib/docker/volumes/nginx-conf/_data:/etc/nginx
depends_on:
- mosquitto1
- mosquitto2
networks:
- backend
mosquitto1:
image: eclipse-mosquitto
container_name: mosquitto1
ports:
- "1886:1883"
networks:
- backend
volumes:
- /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/config:/mosquitto/config
mosquitto2:
image: eclipse-mosquitto
container_name: mosquitto2
ports:
- "1887:1883"
networks:
- backend
volumes:
- /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/config:/mosquitto/config
networks:
backend:
driver: bridge
my nginx.conf
events {
worker_connections 4096; ## Default: 1024
}
stream {
log_format mqtt '$remote_addr [$time_local] $protocol $status $bytes_received '
'$bytes_sent $upstream_addr';
upstream mqtt_cluster {
server 192.168.210.36:1886; #mosquitto1 | 192.168.210.35
server 192.168.210.36:1887; #mosquitto2 | 192.168.210.39
}
server {
listen 1885;
proxy_pass mqtt_cluster;
access_log /var/log/nginx/mqtt_access.log mqtt;
}
}
and my mosquitto.conf
# =================================================================
# Default listener
# =================================================================
# Port to use for the default listener.
listener 1883
allow_anonymous true
persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log
all containers are up and the load balancer is working propertly because it deliver the message to the containers in port 1886 and 1887
I suscribed to them also and can see the messages.
Everytime I publish in the port 1885 the messages delivered to the port 1886 are not printed in port 1885, so in order to see one message I have to execute the publish command twice.
it's like the configuration of the container in the port 1887 is overlapping the port 1886.
I also try leaving only one container and all messages are delivered to the suscriber in port 1885.
Could you tell me if I have something wrong in my docker-composer.yml file please?
