I am getting this problem while pushing the images to registry.
I'm using Ubuntu server inside Ubuntu docker installed, and inside the docker created docker registry container and I have an issue while while registry is integrating with gitlab.
Both gitlab and registry are on same server.
I have created a registry under same domain of gitlab.
no code
Get https://someurl:8000/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I was getting a similar error message:
Get http://localhost:5000/v2/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
when doing a docker push command on my localhost:
docker push localhost:5000/redis:3.0.7
I saw an interesting point there and tried it.
After I edited the /etc/hosts file and commented out the line:
#::1 localhost ip6-localhost ip6-loopback
the issue was gone.
This error encountered due to the destination unreachability problem. It means that the port might be filtered by the local or external firewall or as another answer, the DNS setting of the machine might be the trouble.
Therefore, make sure you're connecting to the indeed destination. My problem was the firewall of the service provider that I got the server, after I created a ssh tunnel, I've logged in as local.
Follow the procedure to check what's the problem.
$ sudo docker login -u mrt https://gitlab.mydomain.com:5050
Error response from daemon: Get https://gitlab.mydomain.com:5050: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Check the connection:
$ nmap -p 5050 gitlab.mydomain.com
Starting Nmap 7.80 ( https://nmap.org ) at 2020-10-12 14:15 +0330
Nmap scan report for gitlab.mydomain.com (x.x.x.x)
Host is up (0.057s latency).
rDNS record for x.x.x.x: x.x.x.x
PORT STATE SERVICE
5050/tcp filtered mmcc
Nmap done: 1 IP address (1 host up) scanned in 0.27 seconds
Create an ssh tunnel:
$ ssh -p 26 -L 5050:gitlab.mydomain.com:5050 user@gitlab.mydomain.com
Then:
$ sudo docker login -u mrt https://localhost:5050
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
I was getting the same error (on Windows):
In my case, it was an issue with the proxy. So I:
This solved the error:
(For private company networks only)
Got the following error intermittently:
net/http request canceled (client.timeout exceeded while awaiting headers)
because DNS was configured in the /etc/docker/daemon.json and the request was failing every time through the public IP i.e. 8.8.8.8. We added another company nameserver and the issue was resolved.
From:
{ "dns" : [ "114.114.114.114" , "8.8.8.8" ] }
To:
{ "dns" : [ "114.114.114.114" , "110.110.110.110", "8.8.8.8" ] }
On Ubuntu 20 I had to open the 5050 port like so: sudo ufw allow 5050/tcp
To see if this is your issue you can run install nmap on gitlab and check to see if port 5050 is open:
apt install nmap
nmap localhost
My results were:
root@gitlab:~# nmap localhost
Starting Nmap 7.80 ( https://nmap.org ) at 2022-07-25 09:04 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.0000070s latency).
Not shown: 988 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
80/tcp open http
443/tcp open https
3000/tcp open ppp
5000/tcp open upnp
5050/tcp open mmcc
8080/tcp open http-proxy
8082/tcp open blackice-alerts
8090/tcp open opsmessaging
9090/tcp open zeus-admin
9100/tcp open jetdirect
So I had port 5050 open.
Then I ran nmap from my local machine
❯ nmap gitlab.example.com
Starting Nmap 7.92 ( https://nmap.org ) at 2022-07-25 11:52 EEST
Nmap scan report for gitlab.example.com (164.92.252.91)
Host is up (0.028s latency).
Not shown: 997 filtered tcp ports (no-response)
PORT STATE SERVICE
22/tcp open ssh
80/tcp open http
443/tcp open https
as you can see we are missing port 5050.
Running sudo ufw allow 5050/tcp on Ubuntu 20 solved the issue for me
Deleting all other DNS server settings except 8.8.8.8 resolved the issue for me.
On mac OS:
On Windows OS, Running Linux Containers with WSL2, i followed the following steps,
For those who use the docker-compose, particularly on ubuntu:
docker-compose push
That means does not include SUDO. I used: sudo docker-compose push and got the error
I had similar failure, but for default docker repository:
ERROR: Get "https://registry-1.docker.io/v2/": net/http: request
canceled while waiting for connection (Client.Timeout exceeded while
awaiting headers)
and
> [internal] load metadata for docker.io/library/openjdk:8-jre:
------
failed to solve: rpc error: code = Unknown desc = failed to solve
with frontend dockerfile.v0: failed to create LLB definition: failed
to do request: Head "https://registry-1.docker.io/v2/library/openjdk/manifests/8-jre":
unable to connect to 52.55.168.20:443. Do you need an HTTP proxy?
Finally I've found this record in my /etc/hosts:
52.55.168.20 registry-1.docker.io
No idea which way it was added (may be it was Docker Desktop setup; I do not remember to do it intendedly), but just deleting or commenting this line solved my problem.
if in the case of network changes:
ERROR: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Solution:
docker network prune
In my case it was IPv6 that caused DNS instability on my GitLab server. Disabling IPv6 on the server helped:
cat <<EOF >> /etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.lo.disable_ipv6=1
EOF
Apply settings.
sudo sysctl --system
I was using wsl in Windows 10 and faced this issue. I know it's not reasonable but everything was ok before changing the Windows password (but I didn't restart the PC). After restarting the PC and running wsl I could pull the image without this error.
