Files needed to run a container docker-compose command

Question

I think I have a bit of a hard time understanding what files do I need to be able to run a container with my Rails app on an empty instance.

I have a docker-compose.prod.yml that I want to run:

version: "3.8"

services:
  db:
    image: postgres
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-default}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-default}
    volumes:
      - ./tmp/db:/var/lib/postgresql/data
  app:
    image: "username/repo:${WEB_TAG:-latest}"
    depends_on:
      - db
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec puma -C config/puma.rb"
    volumes:
      - .:/myapp
      - public-data:/myapp/public
      - tmp-data:/myapp/tmp
      - log-data:/myapp/log
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-default}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-default}
      POSTGRES_HOST: ${POSTGRES_HOST:-default}
  web:
    build: nginx
    volumes:
      - public-data:/myapp/public
      - tmp-data:/myapp/tmp
    ports:
      - "80:80"
    depends_on:
      - app
volumes:
  public-data:
  tmp-data:
  log-data:
  db-data:

So, in the instance, I have the docker-compose.prod.yml file. And since I am passing variables for the environments and the image web tag, I created an .env file with those variables in it as well. Finally, since I am building nginx, I have a folder with the image:

FROM arm64v8/nginx

# インクルード用のディレクトリ内を削除
RUN rm -f /etc/nginx/conf.d/*

# Nginxの設定ファイルをコンテナにコピー
ADD nginx.conf /etc/nginx/myapp.conf

# ビルド完了後にNginxを起動
CMD /usr/sbin/nginx -g 'daemon off;' -c /etc/nginx/myapp.conf

and config file nginx.conf

user  root;
worker_processes  1;

events{
    worker_connections  512;
}

# ソケット接続
http {
  upstream myapp{
    server unix:///myapp/tmp/sockets/puma.sock;
  }
  server { # simple load balancing
    listen 80;
    server_name localhost;

    #ログを記録しようとするとエラーが生じます
    #root /myapp/public;
    access_log /var/log/nginx/access.log;
    error_log  /var/log/nginx/error.log;

    location / {
      proxy_pass http://myapp;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
    }
  }
}

So, docker-compose.prod.yml, nginx directory with the 2 files and the .env file.

When I do: docker-compose -f docker-compose.prod.yml --env-file .env run app rake db:create db:migrate it downloads the postgres and app images but once it starts doing the rake for the db:create db:migrate, I get this error:

Status: Downloaded newer image for user/repo:48
Creating rails-app_db_1 ... done
Creating rails-app_app_run ... done
rake aborted!
No Rakefile found (looking for: rakefile, Rakefile, rakefile.rb, Rakefile.rb)
/usr/local/bundle/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
(See full trace by running task with --trace)

but then when I add the Rakefile, it keeps asking for other dependents files so either I need the whole project itself (clone it from my repo on GitHub) or I am doing this wrong.

Any ideas on what files do I need or if I need to change commands are welcome! Thank you.

Answer 1

When your docker-compose.yml file says

volumes:
  - .:/myapp

It means the contents of the /myapp directory in your image – probably the entire application – are ignored and replaced with whatever's in your host directory. So with this setup you need to copy the entire application source code, or it won't work.

But also:

volumes:
  - public-data:/myapp/public

Your application's static assets are being stored in a Docker named volume. This is hard to transfer between systems, and it will ignore any changes in the underlying image.

---

I'd update a couple of things in this setup, both to avoid most of the volumes and to make things a little easier to work with.

Docker has an internal networking layer, and you can communicate between containers using their Compose service names as host names. (See Networking in Compose in the Docker documentation.) That means you can set the Nginx reverse proxy to talk to the Rails app over normal HTTP/TCP

upstream myapp {
  server http://app:9292; # or the port from your config.ru/puma.rb file
}

This eliminates the need for the tmp-data volume.

In the same way you're building your application code into an image, you can also build the static assets into an image. Update the Nginx image Dockerfile to look like:

# Artificial build stage to include app; https://stackoverflow.com/a/69303997
ARG appimage
FROM ${appimage} AS app

FROM arm64v8/nginx

# Replace the Nginx configuration
RUN rm -f /etc/nginx/conf.d/*
COPY nginx.conf /etc/nginx/nginx.conf

# Copy in the static assets
COPY --from=app /myapp/public /myapp/public

# Use the default CMD from the base image; no need to rewrite it

This removes the need for the public-data volume.

In your main application image, you should declare an ENTRYPOINT and CMD so you don't need to repeat the long-winded command:. For the ENTRYPOINT, I'd suggest a shell script:

#!/bin/sh
# entrypoint.sh

# Remove a stale pid file
rm -f tmp/pids/server.pid

# Run the main container CMD under Bundler
exec bundle exec "$@"

Make sure this file is executable (chmod +x entrypoint.sh) and add it to your repository, maybe in the top-level directory next to your Dockerfile and Gemfile. In the Dockerfile, declare this script as the ENTRYPOINT, and make Puma be the CMD:

ENTRYPOINT ["./entrypoint.sh"]       # must be JSON-array syntax
CMD ["puma", "-C", "config/puma.rb"] # could be shell syntax

The only volume mounts we haven't touched are the database storage and the log directory. For the database storage, a Docker named volume could be appropriate, since you never need to look at the files directly. A named volume is substantially faster on some platforms (MacOS and some cases of Windows) but is harder to transfer between systems. Conversely, for the logs, I'd use a bind mount, since you do generally want to read them directly from the host system.

This reduces the docker-compose.yml file to:

version: "3.8"
services:
  db: { unchanged: from the original question }
  app:
    image: "username/repo:${WEB_TAG:-latest}"
    depends_on:
      - db
    volumes:
      - ./tmp/log:/myapp/log
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-default}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-default}
      POSTGRES_HOST: ${POSTGRES_HOST:-default}
  web:
    image: "username/web:${WEB_TAG:-latest}"
    ports:
      - "80:80"
    depends_on:
      - app

We've removed almost all of the volumes:, and all of the references to host directories except for the database storage and a directory to output the logs. We've also removed the command: override as repeating what's in the Dockerfile. (In other similar SO questions, I might remove unnecessary networks:, container_name:, and hostname: declarations, along with obsolete links: and expose: options.)

If you've done this, you need a way to build your images and push them to the repository. You can have a second Compose file that only describes how to build the images:

# docker-compose.override.yml
# in the same directory as docker-compose.yml
version: '3.8'
services:
  app:
    build: .
  web:
    build:
      context: ./nginx
      args:
        appimage: username/repo:${WEB_TAG:-latest}

Compose's one shortcoming in this situation is that it doesn't know that the one image depends on the other. This means you need to manually build the base image first.

export WEB_TAG=20220305
docker-compose build app
docker-compose build
docker-compose push

This seems like a lot of setup. But having done this, the only thing we need to copy to the new system is the docker-compose.yml file and the production .env settings.

# on the local system
scp docker-compose.yml .env there:

# on the remote system
export WEB_TAG=20220305
docker-compose run app \
  rake db:create db:migrate
docker-compose up -d

Docker will automatically pull the images from the repository if it doesn't have a copy locally. (It's helpful, and required in contexts like Kubernetes, to use a unique image tag per build; docker system prune can clean up old unused images.) If you give the files their default names docker-compose.yml and .env then you don't need to mention them on the command line.