Docker nuget connection timeout

Question

Trying to utilize official jetbrains\teamcity-agent image on Kubernetes. I've managed to run Docker in Docker there but trying to build an ASP.NET Core image with docker build command failes on dotnet restore with

The HTTP request to 'GET https://api.nuget.org/v3/index.json' has timed out after 100000ms.

When I connect to the pod itself and try curling the URL it's super fast. So I assume network is not an issue. Thank for any advice.

Update

Trying to run a simple dotnet restore step from container worked. But not from inside the docker build.

Update 2

I've isolated the problem, it has nothing to do with nuget nor TeamCity. Is network related on the Kubernetes host.

Running simple docker build with this Dockerfile:

FROM praqma/network-multitool AS build
RUN route
RUN ping -c 4 google.com

produces output:

Step 1/3 : FROM praqma/network-multitool AS build
 ---> 3619cb81e582
Step 2/3 : RUN route
 ---> Running in 80bda13a9860
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         172.17.0.1      0.0.0.0         UG    0      0        0 eth0
172.17.0.0      *               255.255.0.0     U     0      0        0 eth0
Removing intermediate container 80bda13a9860
 ---> d79e864eafaf
Step 3/3 : RUN ping -c 4 google.com
 ---> Running in 76354a92a413
PING google.com (216.58.201.110) 56(84) bytes of data.

--- google.com ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 53ms

Pods orchestrated by Kubernetes can access internet normally. I'm using Calico as network layer.

Answer 1

I fix this issue by passing argument --disable-parallel to restore command which Disables restoring multiple projects in parallel.

RUN dotnet restore --disable-parallel

Answer 2

In my case the solution was marked out here

As noted in the comment, "So maybe the issue needs to be fixed by microsoft by changing the default nuget.config inside of mcr.microsoft.com/dotnet/sdk:5.0."

This was my problem. Docker building from sdk:5.0. Solution seems to be doing the job, which is to add a nuget.config file to the root of the solution.

Contents of nuget.config (again, from posts in that issue):

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <config>
    <add key='maxHttpRequestsPerSource' value='10' />
  </config>
</configuration>

Answer 3

i have exactly same behaviour: i have solution with contains several nuget dependencies it build without any issue on local machine. it build without any issue on windows build agent it build without any issue on docker host machine but then i try to build it in build agent in docker - i have a lot of message such following:

Failed to download package 'System.Threading.4.0.11' from 'https://api.nuget.org/v3-flatcontainer/system.threading/4.0.11/system.threading.4.0.11.nupkg'.
  The download of 'https://api.nuget.org/v3-flatcontainer/system.threading/4.0.11/system.threading.4.0.11.nupkg' timed out because no data was received for 60000ms

i can ping and curl page from nuget.org normally from docker container.

so i think this is some special case. i found some info about MTU but i'm not tested it.

UPDATE initial problem may be connect to k8s - my container work inside k8s cluster based on ubuntu 18.04 with flannel ang k8s v1.16 on my local machine (win based) all works without any issue... but it is strange because i have many services that works in this cluster without any problems! (such harbor, graylog, jaeger etc)

UPDATE 2 ok, now i can understand anything. i try to execute

curl https://api.nuget.org/v3/index.json 

and can get file content without any errors

after this i try to run

wget https://api.nuget.org/v3-flatcontainer/system.threading/4.0.11/system.threading.4.0.11.nupkg

and package downloaded successfully

but after i run dotnet restore i still receive errors with timeout

UPDATE 3 i try to reproduce problem not in k8s cluster but in docker locally i run container

docker run -it -v d:/project/test:/mnt/proj teamcity-agent-core3.1 bash

teamcity-buildagent-core3.1 - my image based on jetbrains/teamcity-agent which contains .net core 3.1 sdk.

and then execute command inside interactive session:

dotnet restore test.sln

with failed with following messages:

Failed to download package 'System.Runtime.InteropServices.4.3.0' from 'https://api.nuget.org/v3-flatcontainer/system.runtime.interopservices/4.3.0/system.runtime.interopservices.4.3.0.nupkg'.
   Received an unexpected EOF or 0 bytes from the transport stream.
  The download of 'https://api.nuget.org/v3-flatcontainer/system.text.encoding.extensions/4.3.0/system.text.encoding.extensions.4.3.0.nupkg' timed out because no data was received for 60000ms.
    Exception of type 'System.TimeoutException' was thrown.

Answer 4

I had a similar issue. The mistake I was doing was not specifying the exact dotnet version on the docker image.

FROM mcr.microsoft.com/dotnet/core/sdk AS build

My project targets dotnet 2.2. What I did not know was this was pulling the latest dotnet SDK 3.1. So when the dotnet restore ran, it was timing out.

So this is what I did.

FROM mcr.microsoft.com/dotnet/core/sdk:2.2 AS build

I had to specify a specific version. I'm not sure if this is relation to your problem but I hope it send you in the right direction. Always be explicit with the image version.

Answer 5

I had similar of @NIMROD MAINA and @Anatoly Kryzhanovsky issue when i was using build in docker container from gitlab-runner (docker).

When i run dotnet restore outside docker container. Everything it's work!

Answer 6

In my case it didn't work when nuget.config was inside the project folder.

I put nuget.config in the solution folder (out of the project folder) and it worked again.

Answer 7

For me it was solution setting docker (Windows) to:

• Expose daemon on tcp://localhost:2375 without TLS (true) and
• Use Docker Compose V2 (true)

It's temporary solution, but it works.

Answer 8

Check your DNS settings (A record). Try to type nslookup yourfeeddomain. Make sure that IP address is one and resolved.