Zeppelin Docker Interpreter Configuration

Question

I have created a Zeppelin docker image in my local system and configured the Spark Interpreter through maven repositories and runned the Zeppelin It worked. But when I stop the Docker and runned again the Interpreter binding was gone. How to solve this Issue ? I want that Interpreter binding one-time so that when ever I stop the docker and run again It has to store those interpreter Binding as it is.

Answer 1

You need 3 volumes for persisting configurations, notebooks and logs.
Note: If you added custom interpreters, you need an additional volume for your interpreter binaries.

docker volume create zeppelin-conf
docker volume create zeppelin-notebook
docker volume create zeppelin-logs
docker volume create zeppelin-interpreter

Run the container with above volumes mounted.

docker run -d --restart always -p 8080:8080 -v zeppelin-conf:/zeppelin/conf -v zeppelin-notebook:/zeppelin/notebook -v zeppelin-logs:/zeppelin/logs -v zeppelin-interpreter:/zeppelin/interpreter apache/zeppelin:0.8.1

If you just want to persist configurations you can use following lines:

docker volume create zeppelin-conf
docker run -d --restart always -p 8080:8080 -v zeppelin-conf:/zeppelin/conf apache/zeppelin:0.8.1

Configurations:/zeppelin/conf
Notebooks: /zeppelin/notebook
Logs: /zeppelin/logs
Interpreters: /zeppelin/interpreter

Edit: The /zeppelin directory is the default home directory for docker images. See Dockerfile. Therefore, you don't need to specify ZEPPELIN_NOTEBOOK_DIR, ZEPPELIN_LOG_DIR or ZEPPELIN_INTERPRETER_DIR environment variables.

Answer 2

Mount file into docker run is easy - just pass it into --volume parameter. But in zeppelin case there some parameters pre-configured there, so replace it with empty file is most likely is not what you want achieve. So I could recommend first get that file with default content from container and then mount to it in next run. Please follow step-by step instructions:

1. First we prepare default config for nest runs.

   1. Run default container temporary:

      sudo docker run -d --name zeppelin-test apache/zeppelin:0.8.1
      

   2. And get default config from it:

      mkdir -p conf
      sudo docker zeppelin-test cat /zeppelin/conf/interpreter.json > conf/interpreter.json
      

   Note 1: It will not be used for work, so most parameters unimportant. It need to be done once for setup only!

   Note 2: Because that config populated on start unfortunately you can't obtain it on single run like: sudo docker run --rm apache/zeppelin:0.8.1 cat /zeppelin/conf/interpreter.json

2. Now we can provide use it as bind-mount.

   1. If you use direct docker run method without docker-compose, add option, among others: --volume $(pwd)/conf/interpreter.json:/zeppelin/conf/interpreter.json

   2. But I recommend use docker-compose, so there option placed under volumes: key like - ./conf/interpreter.json:/zeppelin/conf/interpreter.json. Full example:

      version: '3.7'
      
      services:
        zeppelin:
          image: apache/zeppelin:0.8.1
          ports:
            - "7077:7077"
            - "8080:8080"
          volumes:
            - ./logs:/logs
            - ./notebook:/notebook
            - ./conf/interpreter.json:/zeppelin/conf/interpreter.json
          environment:
            ZEPPELIN_NOTEBOOK_DIR: /notebook
            ZEPPELIN_LOG_DIR: /logs
      

      And then just run from that directory:

      docker-compose up -d
      

Answer 3

Interpreter bindings are stored in conf/interpreter.json. Need use external interpreter.json file.