0

Using global ARG variable in instructions like FROM, RUN

for example i wanna use ${CUDA_VERSION} ARG variable in FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION} and libcudnn7=${CUDNN_VERSION}-1+cuda${CUDA_VERSION} in second build stage

but global ARG variable ${CUDA_VERSION} was changed that after pass FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION} 9.0 to 9.0.176

in Ubuntu 18.04, Docker-CE 18.09.04

i was tried many things

  1. Change ARG variable line position in build stage
  2. Copy other ARG variable from original ${CUDA_VERSION} variable
  3. Making .profile for environment variable in first build stage. and in second stage copy .profile file from first stage and apply it using source command
  4. Using ENV variable(but ENV variable disappear when entered other build stage)

example dockerfile and result of build dockerfile as following

Dockerfile

ARG handler_file=handler.py
ARG handler_name=Handler
ARG HANDLER_DIR=/handler
ARG HANDLER_FILE=${HANDLER_DIR}/${handler_file}
ARG HANDLER_NAME=${handler_name}

# Global arguments for Nvidia-docker
ARG CUDA_VERSION=9.0
ARG CUDNN_VERSION=7.4.1.5
ARG UBUNTU_VERSION=16.04

# == MutiStage Build ==
# 1-Stage
FROM python:3.7-alpine

ARG HANDLER_DIR
ARG HANDLER_FILE
ARG HANDLER_NAME
ARG handler_file
ARG handler_name

ARG CUDA_VERSION
RUN echo "${CUDA_VERSION}"

RUN mkdir -p ${HANDLER_DIR}
WORKDIR ${HANDLER_DIR}
COPY . .
RUN touch ${HANDLER_DIR}/__init__.py

# 2-Stage
FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION}

# For Nvidia-Docker
ARG CUDA_VERSION
ARG CUDNN_VERSION

RUN echo "${CUDA_VERSION}"

# Copy directory from 1-stage
ARG HANDLER_DIR
RUN mkdir -p ${HANDLER_DIR}
WORKDIR ${HANDLER_DIR}
COPY --from=0 ${HANDLER_DIR} .

RUN echo "/usr/local/cuda-${CUDA_VERSION}/extras/CUPTI/lib64" > /etc/ld.so.conf.d/cupti.conf

RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    wget \
    tar \
    libgomp1 \
    libcudnn7=${CUDNN_VERSION}-1+cuda${CUDA_VERSION} \
    python \
        python-dev \
        python-numpy \
        python-pip \
        python-setuptools \
        python3 \
        python3-dev \
        python3-numpy \
    python3-pip \
        python3-setuptools \
        python3-tk \
        libgtk2.0-dev \
    ${ADDITIONAL_PACKAGE} \
    && rm -rf /var/lib/apt/lists/*

ENV LD_LIBRARY_PATH /usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH

RUN pip3 --no-cache-dir install --upgrade \
    pip setuptools

RUN pip3 install --upgrdae pip && \
    pip3 install -r requirements.txt

Build Message

...

Step 9/33 : FROM python:3.7-alpine
 ---> 2caaa0e9feab

...

Step 16/33 : RUN echo "${CUDA_VERSION}"
 ---> Running in d057b0fd57a7
9.0

...

Step 21/33 : FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION}
 ---> 2f9810b1b916

...

Step 24/33 : RUN echo "${CUDA_VERSION}"
 ---> Running in dc676c2a2992
9.0.176

...

Step 30/33 : RUN apt-get update && apt-get install -y --no-install-recommends   build-essential     wget    tar     libgomp1    libcudnn7=${CUDNN_VERSION}-1+cuda${CUDA_VERSION}    python         python-dev         python-numpy         python-pip         python-setuptools         python3         python3-dev         python3-numpy   python3-pip         python3-setuptools         python3-tk         libgtk2.0-dev     ${ADDITIONAL_PACKAGE}     && rm -rf /var/lib/apt/lists/*
 ---> Running in 8518fb8d755c

...

E: Version '7.4.1.5-1+cuda9.0.176' for 'libcudnn7' was not found
The command '/bin/sh -c apt-get update && apt-get install -y --no-install-recommends    build-essential     wget    tar     libgomp1    libcudnn7=${CUDNN_VERSION}-1+cuda${CUDA_VERSION}    python         python-dev         python-numpy         python-pip         python-setuptools         python3         python3-dev         python3-numpy   python3-pip         python3-setuptools         python3-tk         libgtk2.0-dev     ${ADDITIONAL_PACKAGE}     && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100

The expected result is docker file build successfully

but ARG variable changed causes the following error E: Version '7.4.1.5-1+cuda9.0.176' for 'libcudnn7' was not found

DongHyun Hwang
  • 1

1 Answers1

0

i was resolve my problem as following

and i posted in issue https://github.com/docker/for-linux/issues/713

# Global arguments for Nvidia-docker
ARG CUDA_VERSION=9.0
ARG CUDNN_VERSION=7.4.1.5
ARG UBUNTU_VERSION=16.04
ARG BACKUP=${CUDA_VERSION}

...

RUN echo "/usr/local/cuda-${BACKUP}/extras/CUPTI/lib64" > /etc/ld.so.conf.d/cupti.conf

RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    wget \
    tar \
    libgomp1 \
    libcudnn7=${CUDNN_VERSION}-1+cuda${BACKUP} \
    python \
        python-dev \
        python-numpy \
        python-pip \
        python-setuptools \
        python3 \
        python3-dev \
        python3-numpy \
    python3-pip \
        python3-setuptools \
        python3-tk \
        libgtk2.0-dev \
    ${ADDITIONAL_PACKAGE} \
    && rm -rf /var/lib/apt/lists/*

...

Dockerfile

# Arguments for Nvidia-Docker
# all combination set in CUDA, cuDNN, Ubuntu is not Incompatible please check REFERENCE OF NVIDIA-DOCKER
# REFERENCE OF NVIDIA-DOCKER 
# https://hub.docker.com/r/nvidia/cuda/

ARG handler_file=handler.py
ARG handler_name=Handler
ARG HANDLER_DIR=/handler
ARG HANDLER_FILE=${HANDLER_DIR}/${handler_file}
ARG HANDLER_NAME=${handler_name}

# Global arguments for Nvidia-docker
ARG CUDA_VERSION=9.0
ARG CUDNN_VERSION=7.4.1.5
ARG UBUNTU_VERSION=16.04
ARG BACKUP=${CUDA_VERSION}

# == MutiStage Build ==
# 1-Stage
# Get watcher - if watcher is uploaded on github, remove this line.
FROM python:3.7-alpine

ARG HANDLER_DIR
ARG HANDLER_FILE
ARG HANDLER_NAME
ARG handler_file
ARG handler_name

ARG BACKUP
ARG CUDA_VERSION
RUN echo "${CUDA_VERSION}"
RUN echo "${BACKUP}"

RUN mkdir -p ${HANDLER_DIR}
WORKDIR ${HANDLER_DIR}
COPY . .
RUN touch ${HANDLER_DIR}/__init__.py

# 2-Stage
FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION}

ARG BACKUP
# For Nvidia-Docker
ARG CUDA_VERSION
ARG CUDNN_VERSION

RUN echo "${CUDA_VERSION}"
RUN echo "${BACKUP}"

# Copy directory from 0-stage
ARG HANDLER_DIR
RUN mkdir -p ${HANDLER_DIR}
WORKDIR ${HANDLER_DIR}
COPY --from=0 ${HANDLER_DIR} .

RUN echo "/usr/local/cuda-${BACKUP}/extras/CUPTI/lib64" > /etc/ld.so.conf.d/cupti.conf

RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    wget \
    tar \
    libgomp1 \
    libcudnn7=${CUDNN_VERSION}-1+cuda${BACKUP} \
    python \
        python-dev \
        python-numpy \
        python-pip \
        python-setuptools \
        python3 \
        python3-dev \
        python3-numpy \
    python3-pip \
        python3-setuptools \
        python3-tk \
        libgtk2.0-dev \
    ${ADDITIONAL_PACKAGE} \
    && rm -rf /var/lib/apt/lists/*

ENV LD_LIBRARY_PATH /usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH

RUN pip3 --no-cache-dir install --upgrade \
    pip setuptools

RUN pip3 install --upgrdae pip && \
    pip3 install -r requirements.txt
DongHyun Hwang
  • 1