Docker Redis - Seeding Data - Cannot execute redis-cli via Dockerfile or via shell script

Question

Have a single Redis instance set up via docker-compose (using my Redis Dockerfile), all working.

Now I'm trying to seed the Redis instance with data after the container has started.

(1) Trying to run redis-cli directly in Dockerfile (that is run by docker-compose)

FROM redis:6.2.5
COPY ./redis.conf /test_dir/
COPY ./seed.txt /test_dir/

ENTRYPOINT ["redis-server", "/test_dir/redis.conf"]
RUN cat /test_dir/seed.txt | redis-cli -a <password> --pipe

this throws Could not connect to Redis at 127.0.0.1:6379: Connection refused

(2) Slightly different Dockerfile, running same command in a shell script that is executed after the container starts:

FROM redis:6.2.5
COPY ./redis.conf /test_dir/
COPY ./seed.txt /test_dir/
COPY ./init.sh /test_dir/

ENTRYPOINT ["redis-server", "/test_dir/redis.conf"]
RUN . /test_dir/init.sh

# init.sh
sleep 10  # ensure Redis server is up and running
cat /test_dir/seed.txt | redis-cli -a <password> --pipe

the shell script is executed, it waits 10 secs and I confirmed the server was up and running BEFORE the data import was triggered, but Redis throws the same error.

Notes:

(1) I've also disabled `protected-mode` and removed `requirepass`, but redis-cli can not connect
(2) when I jump into the container and manually execute the command, it DOES WORK !

So how can I run redis-cli from the Dockerfile or via shell script or what is the recommended way to seed a Docker Redis instance ?

Thanks.

Answer 1

RUN cat /test_dir/seed.txt | redis-cli -a <password> --pipe will execute when you do docker build, at that time ENTRYPOINT ["redis-server", "/test_dir/redis.conf"] still not run as it's only be called when the container start. So, you will surely have error.

As a result, you could use next workaround to do it:

/test_dir/seed.txt:

/test_dir/init.sh:

while :
do
    redis-cli -h redis-svr -p 6379 quit
    if [ $? -eq 0 ]; then
        cat /test_dir/seed.txt | redis-cli -h redis-svr -p 6379 --pipe
        break
    else
        echo "server not ready, wait then retry..."
        sleep 3
    fi
done

docker-compose.yaml:

version: '3'

services:
  redis-svr:
    image: redis
  redis-cli:
    image: redis
    volumes:
      - /test_dir:/test_dir
    entrypoint: sh -c /test_dir/init.sh

Execution:

$ docker-compose up
WARNING: Found orphan containers (20210910_redis_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Starting 20210910_redis-svr_1 ... done
Starting 20210910_redis-cli_1 ... done
Attaching to 20210910_redis-cli_1, 20210910_redis-svr_1
redis-cli_1  | Could not connect to Redis at redis-svr:6379: Connection refused
redis-cli_1  | server not ready, wait then retry...
redis-svr_1  | 1:C 11 Sep 2021 05:55:33.872 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis-svr_1  | 1:C 11 Sep 2021 05:55:33.872 # Redis version=6.2.5, bits=64, commit=00000000, modified=0, pid=1, just started
redis-svr_1  | 1:C 11 Sep 2021 05:55:33.872 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis-svr_1  | 1:M 11 Sep 2021 05:55:33.873 * monotonic clock: POSIX clock_gettime
redis-svr_1  | 1:M 11 Sep 2021 05:55:33.879 * Running mode=standalone, port=6379.
redis-svr_1  | 1:M 11 Sep 2021 05:55:33.879 # Server initialized
redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * Loading RDB produced by version 6.2.5
redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * RDB age 266 seconds
redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * RDB memory usage when created 0.77 Mb
redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * DB loaded from disk: 0.010 seconds
redis-svr_1  | 1:M 11 Sep 2021 05:55:33.890 * Ready to accept connections
redis-cli_1  | OK
redis-cli_1  | All data transferred. Waiting for the last reply...
redis-cli_1  | Last reply received from server.
redis-cli_1  | errors: 0, replies: 0
20210910_redis-cli_1 exited with code 0

Explain:

redis-svr will start a server there in one container, redis-cli will start another container, it will frist call init.sh, the init.sh will try to link to redis server to see if server really start or not with redis-cli -h redis-svr -p 6379 quit. If not, it will wait sometime and retry, if server already start, then it could call the client command to import the initial data to server.

EDIT20210912 based on OP's comment to use one container:

Folder structure:

$ tree
.
├── docker-compose.yaml
├── Dockerfile
├── init.sh
├── my-entrypoint.sh
└── seed.txt

docker-compose.yaml:

version: '3'

services:
  redis-svr:
      build: ./

Dockerfile:

FROM redis:6.2.5
COPY . /tmp
RUN chmod -R 777 /tmp
ENTRYPOINT ["/tmp/my-entrypoint.sh"]
CMD ["redis-server"]

init.sh:

while :
do
    redis-cli quit
    if [ $? -eq 0 ]; then
        echo "Server ready now, start to import data ..."
        cat ./seed.txt | redis-cli --pipe
        break
    else
        echo "Server not ready, wait then retry..."
        sleep 3
    fi
done

my-entrypoint.sh:

#!/bin/sh
/tmp/init.sh &
docker-entrypoint.sh $1

Execution:

$ docker-compose up
Recreating test_dir_redis-svr_1 ... done
Attaching to test_dir_redis-svr_1
redis-svr_1  | Could not connect to Redis at 127.0.0.1:6379: Connection refused
redis-svr_1  | Server not ready, wait then retry...
redis-svr_1  | 7:C 12 Sep 2021 08:36:05.512 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis-svr_1  | 7:C 12 Sep 2021 08:36:05.512 # Redis version=6.2.5, bits=64, commit=00000000, modified=0, pid=7, just started
redis-svr_1  | 7:C 12 Sep 2021 08:36:05.512 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis-svr_1  | 7:M 12 Sep 2021 08:36:05.513 * monotonic clock: POSIX clock_gettime
redis-svr_1  | 7:M 12 Sep 2021 08:36:05.515 * Running mode=standalone, port=6379.
redis-svr_1  | 7:M 12 Sep 2021 08:36:05.515 # Server initialized
redis-svr_1  | 7:M 12 Sep 2021 08:36:05.515 * Ready to accept connections
redis-svr_1  | OK
redis-svr_1  | Server ready now, start to import data ...
redis-svr_1  | All data transferred. Waiting for the last reply...
redis-svr_1  | Last reply received from server.
redis-svr_1  | errors: 0, replies: 0

This define customized entrypoint to let init.sh have chance to be executed before redis server run.

Answer 2

this is another solution that works, although @atline solution is more elegant.

Dockerfile

FROM redis:6.2.5-alpine3.14
RUN mkdir /foo
COPY ./redis.conf /foo/
COPY ./seed.txt /foo/
COPY ./init.sh /foo/

ENTRYPOINT /bin/sh -c "redis-server /foo/redis.conf" & sleep 2s \
&& source /foo/init.sh

init.sh

# seed db
cat /foo/seed.txt | redis-cli -a <pw_if_applicable> --pipe
# save .rdb dump
redis-cli -a <pw_if_applicable> save
# shutdown server
redis-cli -a <pw_if_applicable> shutdown
# restart server
redis-server "/foo/redis.conf"

This works and uses the provided redis.conf, hence it seeds the data into the the .rdb that you have defined in the config.

Answer 3

https://docs.docker.com/config/containers/multi-service_container/ It's very simple try mine init.sh

#!/bin/bash
set -m
redis-server --include /etc/redis.conf &
sleep 5
cat /data/user_data | redis-cli --pipe
fg %1