1

I'm currently trying to setup Gitlab-runner with Docker-machine executor in AWS behind a proxy. Somehow docker-machine can not properly connect to the spawned machines. Here is what i get:

ubuntu@ip-42-1-0-44:~$ sudo -i docker-machine ls
NAME                                                       ACTIVE   DRIVER      STATE     URL                    SWARM   DOCKER    ERRORS
runner-eu3khqy-gitlab-docker-machine-1596546765-ee5149ca   -        amazonec2   Running   tcp://42.1.0.36:2376           Unknown   Unable to query docker version: Cannot connect to the docker engine endpoint

ubuntu@ip-42-1-0-44:~$  eval $(sudo -i docker-machine env runner-eu3khqy-gitlab-docker-machine-1596546765-ee5149ca)
Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host "42.1.0.36:2376": dial tcp 42.1.0.36:2376: connect: connection refused
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which might stop running containers.

More details:
So far i was successful in starting the Gitlab-runner, which is trying to spawn machines. Both, the machine where the runner is on (Bastion) and the machines being spawn, need the proxy to be set in order to connect to the internet. Therefore, i configured cntlm on the Bastion. And gitlab-runner in /etc/systemd/system/gitlab-runner.service.d/http-proxy.conf to work with that proxy. So far so good.

In order for docker to work, i further configured /etc/systemd/system/docker.service.d/http-proxy.conf

[Service]
Environment="HTTP_PROXY=http://127.0.0.1:3128/"
Environment="HTTPS_PROXY=http://127.0.0.1:3128/"

with 3128 being the port cntlm is listening on.

To spawn machines i set config.toml under MachineOptions

"engine-env=HTTP_PROXY=http://proxy.net:8080",
"engine-env=HTTPS_PROXY=http://proxy.net:8080",
"engine-env=NO_PROXY=localhost, 127.0.0.*, 10.*, 42.*, 192.168.*,

which is getting visible under => /root/.docker/machine/machines/runner-?/config.json

"Env": [
    "HTTP_PROXY=http://proxy.net:8080",
    "HTTPS_PROXY=http://proxy.net:8080",
    "NO_PROXY=localhost, 127.0.0.*, 10.*, 42.*, 192.168.*,
],

Debugging:
So when i used docker-machine ssh to connect to the machine, i couldn't find docker being run nor being installed. This is where i assumed, the Proxy variables would not be properly passed to the machine.

Question: What am I doing wrong? How to properly pass proxy variables to the machines?

Update: To narrow things down, i did a manual machine creation:

sudo -i docker-machine --debug create -d amazonec2 -amazonec2-region eu-central-1 -amazonec2-instance-type t2.micro -amazonec2-subnet-id subnet-XXXXX -amazonec2-vpc-id vpc-XXXXXX --amazonec2-use-private-address --engine-env http_proxy=http://proxy.net:8080 --engine-env https_proxy=http://proxy.net:8080 --engine-env no_proxy=localhost,127.0.0.*,10.*,42.*,192.168.* --engine-env HTTP_PROXY=http://proxy.net:8080 --engine-env HTTPS_PROXY=http://proxy.net:8080 --engine-env NO_PROXY=localhost,127.0.0.*,10.*,42.*,192.168.* docker-compose-test2

In the resulting logs i get the timeout at sudo apt-get update.
With several trials and connecting to the machine via ssh, i'm pretty sure that the environment variables passed via engine-env are not properly set. That is, despite me checking the config via docker-machine inspect, where everything seems fine.
Now i wonder, whether this could even be an issue specific to the amazonec2 driver?

Flo Sparrow
  • 31
  • 3

0 Answers0