Spark docker image

Asked Nov 30 '18 at 12:55

Active Nov 30 '18 at 13:13

Viewed 802 times

I want to build a spark 2.4 docker image.I follow the steps as per the link

The command that i run to build the image

./bin/docker-image-tool.sh -t spark2.4-imp build

Here is the output i get. Its stuck at that fetch forever

Any clue?

UPDATE:

Same is the issue when i execute the following command

docket build -t spark2.4-imp -f kubernetes/dockerfiles/spark/Dockerfile .

edited Nov 30 '18 at 13:13

asked Nov 30 '18 at 12:55

Manikandan Kannan

8,684
15
44
65

Are you working behind a (corporate) proxy? – Sven Hakvoort Nov 30 '18 at 12:58
@SvenHakvoort Not really. In fact, when i do a wget, it works – Manikandan Kannan Nov 30 '18 at 13:10
That does not necessarily mean you are not behind a proxy. I had the same issue in my place of work, I could wget from my machine, but the docker daemon did not use the proxy which cause it to be unable to connect to the internet.Could you check for sure if you are behind a proxy? (See: https://askubuntu.com/questions/831266/find-proxy-server-using-command-line) – Sven Hakvoort Nov 30 '18 at 13:12
I tried commands in the link that you provided. None of them fetch any result. Also the wget output does not indicate that there is a proxy (going by the explanation again in the link you gave) – Manikandan Kannan Nov 30 '18 at 13:22
You have done all the prerequisites (mainly DNS config I am curious about)? – Sven Hakvoort Nov 30 '18 at 13:25
I am not familiar about the prerequisites. Can you thrown some light? – Manikandan Kannan Nov 30 '18 at 13:29
In the link you mentioned it lists some prerequisites: https://spark.apache.org/docs/latest/running-on-kubernetes.html#prerequisites – Sven Hakvoort Nov 30 '18 at 13:29
Are those required for creating the image itself? Most of them are for running Spark on K8S and may not be for building the image itself – Manikandan Kannan Nov 30 '18 at 13:32
Yes that is correct, I assumed you are using kubernetes but that assumption i wrong? – Sven Hakvoort Nov 30 '18 at 13:35
I will eventually use but i am stuck at building the docker image itself which has got nothing to do with K8S? – Manikandan Kannan Nov 30 '18 at 13:41
Have you tried completely restarting docker? (So not only the build process, but the whole docker daemon) – Sven Hakvoort Nov 30 '18 at 13:43
I could not figure out what could be the issue in my office network. Finally, i built the same on AWS instance and pushed to docker hub – Manikandan Kannan Dec 19 '18 at 04:10

Spark docker image

0 Answers0