How to simulate AWS S3 in docker-compose using MinIO?

Question

I have an application server that must fetch data from AWS S3, e.g. https://my-bucket.s3.us-east-1.amazonaws.com/assets/images/557a84a8-bd4b-7a8e-81c9-d445228187c0.png

I want to test this application server using docker-compose.

I can spin up an MinIO server quite easily, but how do I configure things so that my application accesses the local MinIO server as if it were the AWS one?

I am using the standard .NET AWS SDK and I do not want to change my application code for testing (this would defeat the point of the tests).

What I have so far:

version: '3.9'
services:
  s3:
    image: quay.io/minio/minio:RELEASE.2022-08-13T21-54-44Z
    command: minio server /data
    ports:
      - "9000:9000"
    environment:
      MINIO_ROOT_USER: minio
      MINIO_ROOT_PASSWORD: minio123
      MINIO_ACCESS_KEY: minio_access_key
      MINIO_SECRET_KEY: minio_secret_key
    restart: always 

  server:
    image: server:latest
    ports:
      - "8080:8080"
    environment:
      AWS_ACCESS_KEY_ID: minio_access_key
      AWS_SECRET_ACCESS_KEY: minio_secret_key
    depends_on:
      s3:
        condition: service_started

larsks · Answer 1 · 2022-08-18T01:01:16.263

You can set a network alias on your s3 container, to make it available as my-bucket.s3.us-east-1.amazonaws.com.
You can tell minio server to recognize name-based buckets rooted at s3.us-east-1.amazonaws.com by setting the MINIO_DOMAIN environment variable (see the Server Confihttps://docs.docker.com/compose/compose-file/compose-file-v3/#networksguration Guide
You can change the port on which minio listens by setting the --address command line option (or by putting a proxy in front of it)

That gets you:

services:
  s3:
    image: quay.io/minio/minio:RELEASE.2022-08-13T21-54-44Z
    command: minio server /data
    ports:
      - "9000:9000"
    environment:
      MINIO_ROOT_USER: minio
      MINIO_ROOT_PASSWORD: minio123
      MINIO_ACCESS_KEY: minio_access_key
      MINIO_SECRET_KEY: minio_secret_key
      MINIO_DOMAIN: s3.us-east-1.amazonaws.com
    restart: always
    networks:
      default:
        aliases:
          - my-bucket.s3.us-east-1.amazonaws.com

This will almost work: your bucket would be available at http://my-bucket.s3.us-east-1.amazonaws.com:9000. If you want to make it available at https://my-bucket.s3.us-east-1.amazonaws.com, you would need to set up an SSL terminating proxy in front of it (something like Traefik, Nginx, etc), and you would need to create and install the necessary certificates so that your client trusts the server.

Hopefully this is enough to point you in the right direction!

This is good, it's just a shame it doesn't support port 443 out of the box, since the vast majority of S3-compatible tools expect it on port 443. You basically have to do https://min.io/docs/minio/linux/integrations/setup-nginx-proxy-with-minio.html as well. — rjh, Nov 02 '22 at 23:36

How to simulate AWS S3 in docker-compose using MinIO?

1 Answers1