5

I'm a bit new to Docker and I'm trying to copy resources from my cloud bucket to my instance created with a docker image. I use gsutil with the following in my Dockerfile

# Install Google Cloud tools - Debian https://cloud.google.com/storage/docs/gsutil_install#deb
ENV CLOUD_SDK_REPO="cloud-sdk-stretch"
RUN echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | \
    tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
    apt-get update && apt-get install -y google-cloud-sdk

# Setup Google Service Account
COPY service-account.json /etc/
ENV GOOGLE_APPLICATION_CREDENTIALS="/etc/service-account.json"

RUN gcloud auth activate-service-account --key-file=${GOOGLE_APPLICATION_CREDENTIALS}

# Copy the last updated ssl config
RUN gsutil cp -r gs://my-project.appspot.com/docker/etc/letsencrypt /etc/ && \
    gsutil cp -r gs://my-project.appspot.com/docker/etc/apache2/sites-available /etc/apache2/

When I run this on my machine locally, the files get copied correctly with gsutil. (They exist when I run the docker image)

When I deploy to Google Container Registry and Use the docker image on a GCE instance the files don't exist on the running docker image.

I can see from the google build logs that the gsutil appears to be working correctly and is copying the files (during the build process).

What am I doing wrong? Is this a bug?

Any help appreciated!

aaronvargas
  • 12,189
  • 3
  • 52
  • 52

2 Answers2

2

I suspect the files are in /etc/ but it's not clear from the snippet what your image does when you run it.

The RUN steps you show above are run only when the image is built. RUN is used to run steps needed to install and build the software that runs in the container. Run steps are not run when you create a container from the image.

So, if the files are copied correctly during docker build, they'll be present when that image is run. Where are you building the image? Once the image is built, deploying anywhere (including to GCR) and then running the image, won't affect the steps your show (because they were run during image creation).

From what you show, it's not clear what happens when you run the image. If this is the entirety of your Dockerfile, then nothing will happen when you docker run.

NB Your approach has security implications. Anyone who has the image can access your service account key. For data including keys (possibly also /letsencrypt/) and your config, it's good practice to reference these at docker run time and commonly using volumes|mounts.

Do I assume that you're looking to run Apache with this image?

DazWilkin
  • 32,823
  • 5
  • 47
  • 88
  • Hmmm... You are right, I ran this again on a fresh VM and the files Are indeed now in /etc/. I think the issue was that I was doing a 'reset' on the VM instead of a stop/start. I thought that wiped the filesystem back to original state and would pull the latest Docker image (I'm using latest tag). I believe this was causing a stale image (pre gsutil changes) to be used instead of my updated one. Other times, 'reset' has appeared to work correctly. There are some deeper Docker nuances about iterating on the build and re-deploying, it got out of sync and I don't fully understand the layers... – aaronvargas Aug 14 '18 at 16:05
  • As far as security goes, I figure that the container and the gsutil files are both private and secured under the same access model in GCE. I recognize that if I were sharing the image to a wider audience than would have access to the service-account then that would be leaking access. – aaronvargas Aug 14 '18 at 16:15
  • Yes, it's secure within GCP but.... your colleagues and your future self may forget the precious cargo and inadvertently share the image (and thus the secret); you'll diligently cycle the keys every x-days and your image will become borked. I continue to advise against this approach. – DazWilkin Aug 14 '18 at 22:11
1

I suspect you're running into the issue described here:

`gsutil cp` from Storage to compute instance running container doesn't copy files

The alias set up for gsutil will run it in a spun-up container, whose file system is separate from the host. Using this alias, if you copy something to /etc/foo/bar, then look for /etc/foo/bar on host filesystem, it won't be present. The issue linked above contains some workarounds.

mhouglum
  • 2,468
  • 13
  • 21