How to build Docker Images with Dockerfile behind HTTP_PROXY by Jenkins?

Question

Building Docker images works in a desktop without a problem. Installing Node.js NPM dependencies work as usual. However, when using a continuous integration server such as Jenkins that is hosted behind a corporate proxy, the build Docker Images fail.

Node.js NPM Dependencies

While building Node.js pacakges, the command npm install fails when it cannot connect to GIT while cloning GIT dependencies.

e1ce5e8407d1: Already exists
Status: Image is up to date for node:0.10.33
 ---> e1ce5e8407d1
Step 1 : RUN mkdir -p /usr/src/app
 ---> Using cache
 ---> 965cad0c68b0
Step 2 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 4c498f0c07e9
Step 3 : COPY package.json /usr/src/app/
 ---> b0662a8275fb
Removing intermediate container 5aca20551452
Step 4 : RUN npm install
 ---> Running in 7ccf9e5362af
npm WARN package.json newww@2.0.0 No README data
npm WARN package.json Dependency 'async-cache' exists in both dependencies and devDependencies, using 'async-cache@^0.1.5' from dependencies
npm ERR! git clone https://github.com/npm/npm2es.git Cloning into bare repository '/root/.npm/_git-remotes/https-github-com-npm-npm2es-git-60a75edb'...
npm ERR! git clone https://github.com/npm/npm2es.git fatal: unable to access 'https://github.com/npm/npm2es.git/': Failed to connect to github.com port 443: Connection timed out

Java Maven, Ruby, Go Docker Images with Dependencies

The same occurs when building Java, Ruby or Go containers, where dependencies are located in repository servers across your corporate Proxy server.

Knowing that you can configure Docker with HTTP_PROXY environment variable, how to properly configure Docker to properly build images in CI environments?

Why not just do the docker build locally and have the CI test the docker image? — generalhenry, Jan 02 '15 at 22:18
Because we have an internal private Docker Registry, private NPM Registry with Proxy, private Maven repositories and proxies. All of those are properly setup for the current CI/CD environments (Not only public cloud, but we do have multiple private cloud in distributed Data Centers)... So, Engineers don't know where the images will be built, but we do have to specify the HTTP_PROXY variable manually or through some kind of automation. Another reason is that some services also require external HTTP access in order to retrieve dependencies during build and runtime. — Marcello DeSales, Jan 02 '15 at 22:31
This is a common requirement and the solution is using `--build-arg`. You can check the [docs](https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg) too. — therealak12, Sep 30 '21 at 09:38

score 64 · Accepted Answer · answered Aug 13 '15 at 11:58

64

Note: Docker 1.9 might help solve this:

"Issue 14634": Builder - Build-time argument passing (e.g., HTTP_PROXY)
"PR 15182": Support for passing build-time variables in build context

Usage (proposed):

docker build --build-arg http_proxy=http://my.proxy.url  --build-arg foo=bar <<MARK
FROM busybox
RUN <command that need http_proxy>
ARG --description="foo's description" foo
USER $foo
MARK

answered Aug 13 '15 at 11:58

VonC

1,262,500
529
4,410
5,250

Thanks for providing it... I've been following the development of that and I can't wait... – Marcello DeSales Aug 13 '15 at 16:20
1

It's out. We're at Docker 1.9.1 now. – taco Dec 19 '15 at 03:32
I've tested and it is awesome! – Marcello DeSales Dec 29 '15 at 06:06
1

@MarcellodeSales Indeed. I have been using it for a few weeks now, and it does simplifies the proxy management. – VonC Dec 29 '15 at 06:09
1

Yeah, there's an open issue to support it on docker-compose! That makes it a full support! :) – Marcello DeSales Dec 29 '15 at 16:43
@MarcellodeSales Right: https://github.com/docker/compose/issues/2111 I suppose. I will be monitoring it. – VonC Dec 29 '15 at 17:38

jeremysprofile · Answer 2 · 2020-11-26T18:33:57.477

Docker has multiple ways to set proxies that take effect at different times.

If your docker build has to retrieve a base image through a proxy, you'll want to specify build-args:

docker build --build-arg HTTP_PROXY=$http_proxy \
--build-arg HTTPS_PROXY=$http_proxy --build-arg NO_PROXY="$no_proxy" \
--build-arg http_proxy=$http_proxy --build-arg https_proxy=$http_proxy \
--build-arg no_proxy="$no_proxy" -t myContainer /path/to/Dockerfile/directory

where $http_proxy and $no_proxy were set in my bashrc. I used both HTTP_PROXY and http_proxy because different utilities will check different variables (curl checks both, wget only checks the lowercase ones, etc).

If your docker build has a RUN curl/wget/etc command that has to go through the proxy, you'll need to specify an environment variable inside your docker image:

ENV https_proxy=http://proxy-us02.company.com:8080
ENV http_proxy=http://proxy-us02.company.com:8080
ENV HTTP_PROXY=http://proxy-us02.company.com:8080
ENV HTTPS_PROXY=http://proxy-us02.company.com:8080
ENV no_proxy="localhost,localdomain,127.0.0.1,etc"
ENV NO_PROXY="localhost,localdomain,127.0.0.1,etc"

If you don't want this environment variable inside your image at runtime, you can remove all these at the end:

RUN unset http_proxy https_proxy no_proxy HTTP_PROXY HTTPS_PROXY NO_PROXY

See [this question](https://unix.stackexchange.com/questions/212894/whats-the-right-format-for-the-http-proxy-environment-variable-caps-or-no-ca#:~:text=at%2018%3A12-,Unlike%20basically%20all%20conventional%20environment%20variables%20used%20by%20many%20applications,found%20out%20that%20docker%2017.04.) Basically, some applications use uppercase, some use lowercase. — jeremysprofile, Nov 03 '21 at 14:18

score 14 · Answer 3 · answered Jan 02 '15 at 22:25

Docker Daemon HTTP Proxy

A lot of documentation is available about setting up the HTTP_PROXY environment variable for Docker's daemon. The environment variable is only available when running containers, so it won't help us here.

Solution in Dockerfile

Although setting up the environment variable HTTP_ENV or http_env in the Dockerfile might help, it does not help our cause either.

ENV http_proxy http://proxy.mycompany.com:80

The reason why is that each specific service only honors HTTP Proxy setting in a different way. The way I could solve is below.

NPM: NPM requires setting up the HTTP_PROXY variable using a CLI command.
GIT: GIT requires setting up the HTTP_PROXY variable using a CLI command as well.
MAVEN: MVN command requires setting up the HTTP_PROXY as an XML file under the user's directory at ~/.m2/settings.xml. For Docker, you can add it to the root's "/root/.m2/settings.xml" directory (unsafe, development-only), or to the Dockerfile's user's home directory.

For instance, running an application using Dockerfile, I can build an image using the following Dockerfile:

FROM node:0.10.33

# Prepare
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Use the cache for dependencies
COPY package.json /usr/src/app/

# If building behind an http_proxy, set them for git and npm
RUN git config --global http.proxy http://qypprdproxy02.ie.company.net:80 && \
    npm config set proxy http://qypprdproxy02.ie.company.net:80 && \
    npm config set https-proxy http://qypprdproxy02.ie.company.net:80

# Install dependencies
RUN npm install

# Copy all the source
COPY . /usr/src/app

# Execute the dev steps
COPY ./numbat-config.example.js /usr/src/app/numbat-config.js
COPY ./.env.example /usr/src/app/.evn
RUN touch /usr/src/app/config.admin.js

Note that I have configured both GIT and NPM using their CLI command to explicitly take the proxy settings before running the NPM install command. That way, both NPM and GIT dependencies will be automatically retrieved and cloned, respectively.

The result of building an image with this Dockerfile works as expected:

[root@pppdc9prd6dq newww]# fig build
...
...
Building npmregistryserver...
 ---> Using cache
 ---> 965cad0c68b0
Step 2 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 4c498f0c07e9
Step 3 : COPY package.json /usr/src/app/
 ---> ae8ff7861246
Removing intermediate container ba1d7b8c9963
Step 4 : RUN npm config set proxy http://qypprdproxy02.ie.company.net:80 &&     npm config set https-proxy http://qypprdproxy02.ie.company.net:80 &&     npm install
 ---> Running in aa6e05d9c7a4
npm WARN package.json newww@2.0.0 No README data
npm WARN package.json Dependency 'async-cache' exists in both dependencies and devDependencies, using 'async-cache@^0.1.5' from dependencies
npm WARN deprecated extend@1.1.3: Please update to the latest version.

> v8flags@1.0.8 install /usr/src/app/node_modules/gulp/node_modules/v8flags
> node fetch.js


> hiredis@0.1.17 install /usr/src/app/node_modules/hiredis
> node-gyp rebuild

make: Entering directory '/usr/src/app/node_modules/hiredis/build'
  CC(target) Release/obj.target/hiredis/deps/hiredis/hiredis.o
  CC(target) Release/obj.target/hiredis/deps/hiredis/net.o
  CC(target) Release/obj.target/hiredis/deps/hiredis/sds.o
  CC(target) Release/obj.target/hiredis/deps/hiredis/async.o
  AR(target) Release/obj.target/deps/hiredis.a
  COPY Release/hiredis.a
  CXX(target) Release/obj.target/hiredis/src/hiredis.o
  CXX(target) Release/obj.target/hiredis/src/reader.o
  SOLINK_MODULE(target) Release/obj.target/hiredis.node
  SOLINK_MODULE(target) Release/obj.target/hiredis.node: Finished
  COPY Release/hiredis.node
make: Leaving directory '/usr/src/app/node_modules/hiredis/build'
npm WARN engine hawk@0.10.2: wanted: {"node":"0.8.x"} (current: {"node":"0.10.33","npm":"2.1.11"})

> pngcrush-bin@1.0.0 postinstall /usr/src/app/node_modules/imagemin-pngcrush/node_modules/pngcrush-bin
> node lib/install.js

     fetch : https://raw.githubusercontent.com/imagemin/pngcrush-bin/v1.0.0/vendor/linux/pngcrush


✔ pre-build test passed successfully!

> dtrace-provider@0.3.1 install /usr/src/app/node_modules/npm-typeahead/node_modules/restify/node_modules/dtrace-provider
> scripts/install.js

npm WARN engine cryptiles@0.1.3: wanted: {"node":"0.8.x"} (current: {"node":"0.10.33","npm":"2.1.11"})
npm WARN engine sntp@0.1.4: wanted: {"node":"0.8.x"} (current: {"node":"0.10.33","npm":"2.1.11"})
npm WARN engine boom@0.3.8: wanted: {"node":"0.8.x"} (current: {"node":"0.10.33","npm":"2.1.11"})
npm WARN engine hoek@0.7.6: wanted: {"node":"0.8.x"} (current: {"node":"0.10.33","npm":"2.1.11"})
npm WARN cannot run in wd newww@2.0.0 gulp build (wd=/usr/src/app)
newww-metrics@1.0.0 node_modules/newww-metrics

murmurhash@0.0.2 node_modules/murmurhash

npm-humans@2.0.1 node_modules/npm-humans

leven@1.0.1 node_modules/leven

chunk@0.0.2 node_modules/chunk

npm-expansions@1.14.0 node_modules/npm-expansions

similarity@1.0.1 node_modules/similarity

truncate@1.0.4 node_modules/truncate

This properly worked as expected and you can have a CI/CD environment behind an http proxy to rebuild images based on this Dockerfile.

This worked, but it prevents the Dockerfile from building on other devices, I fear. — K.., Mar 19 '15 at 15:53
That's exactly what the question/answer is about: if you have an CI/CD environment where Docker images are built by separate boxes!!! Your app can still run normally by replacing or completely unsetting the proxy settings at runtime by mounting a new .npmrc that removes the setting while you do "docker run". ;) — Marcello DeSales, Mar 20 '15 at 21:42

score 10 · Answer 4 · answered Aug 08 '18 at 06:11

10

Starting with Docker 17.07 you can alternatively use the Docker Client configuration file for providing the proxy configuration centrally:

https://docs.docker.com/network/proxy/#configure-the-docker-client

answered Aug 08 '18 at 06:11

desolat

4,123
6
36
47

1

I tried with `--build-arg` and `ENV` variable but doesn't seems to work. Only this way seems to be working with `Ubuntu 18.04` image for me. Thanks for sharing. I created a file `~/.docker/config.json` with proxy urls as suggested in the post. – Dan Jul 26 '21 at 05:50
All other answers using environment variables or build args are deprecated. This method (configuring docker client) is the right way to solve proxy issues today ! – Donatello Dec 09 '21 at 10:14

danday74 · Answer 5 · 2015-07-16T16:59:00.023

4

We are doing ...

ENV http_proxy http://9.9.9.9:9999
ENV https_proxy http://9.9.9.9:9999

and at end of dockerfile ...

ENV http_proxy ""
ENV https_proxy ""

This, for now (until docker introduces build env vars), allows the proxy vars to be used for build without publicly exposing them

edited Jul 16 '15 at 16:59

answered Jul 16 '15 at 16:40

danday74

52,471
49
232
283

When docker build is running from inside Visual Studio, this seems to be the only way I can get it to recognize the proxy settings. I tried with a NuGet.Config file and setting http_proxy and https_proxy environment variables, but none of these worked. I had to add if after the build statement. In my case it followed "FROM microsoft/dotnet:2.1-sdk AS build". Thanks! – Nimblejoe Oct 23 '18 at 22:20

senthil sivasamy · Answer 6 · 2018-05-18T16:05:39.743

0

I had a problem when corporate network was not allowing to download and setup docker image so n/w gave http proxy information. while running docker image build I passed the variable and it worked without any issues.

  docker build  --build-arg http_proxy="http://userid:pwd@iaisystem.com:8080" - < Dockerfile

edited May 18 '18 at 16:05

answered May 18 '18 at 16:00

senthil sivasamy

352
3
7

score 0 · Answer 7 · answered Jul 26 '18 at 19:03

0

You can use a transparent proxy, as described in:

https://jpetazzo.github.io/2014/06/17/transparent-squid-proxy-docker/

docker run --net host jpetazzo/squid-in-a-can
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to 3129

answered Jul 26 '18 at 19:03

Atila Romero

359
3
7

score 0 · Answer 8 · answered Jan 12 '23 at 12:47

I had a similar issue where docker build was unable to pull the base image from a registry hosted on Amazon ECR. I found that adding proxy configuration to the docker build command line, the docker file or my shell made no difference.

The fix that worked for me was to change the environment for the docker daemon. I am using a CentOS box, and on that system there is config directory /etc/systemd/system/docker.service.d and in there a http-proxy.conf file.

I edited that file to put the correct proxy configuration, and restarted the docker daemon, and then my docker build started working.

[Service]
Environment="HTTP_PROXY=http://<hostname>:8080"
Environment="HTTPS_PROXY=http://<hostname>:8080"
Environment="NO_PROXY=<suffix>,<another suffix>"

This does not provide an answer to the question. Once you have sufficient [reputation](https://stackoverflow.com/help/whats-reputation) you will be able to [comment on any post](https://stackoverflow.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/33616744) — rastasheep, Jan 18 '23 at 09:50

How to build Docker Images with Dockerfile behind HTTP_PROXY by Jenkins?

Node.js NPM Dependencies

Java Maven, Ruby, Go Docker Images with Dependencies

8 Answers8

Docker Daemon HTTP Proxy

Solution in Dockerfile

Linked