Deploy Scio jobs with custom containers

Question

I would like to deploy a Scio Job on Dataflow with custom containers https://beam.apache.org/documentation/runtime/environments/

Is it possible to package the app with sbt-pack for example, and extend the base image to build an image with all the things? I succeed to do this with Flex Templates but there is some limitations (we can't launch an another job from a template via a new ScioContext) so I would like to do the same thing with custom containers but I don't know how to do. I presume I need to copy my jar and lib but I don't know if the entrypoint must be override or other. Here my Dockerfile:

FROM apache/beam_java11_sdk:2.49.0

RUN apt-get update && apt-get -y install \
    apt-transport-https locales-all libpng16-16 libxinerama1 libgl1-mesa-glx libfontconfig1 libfreetype6 libxrender1 \
    libxcb-shm0 libxcb-render0 adduser cpio findutils \
    procps \
    && apt-get -y install libreoffice --no-install-recommends \
    && apt-get install -y libreoffice-java-common \
    && rm -rf /var/lib/apt/lists/* 


ARG APP_NAME="my-app"
ARG APP_HOME="/app/$APP_NAME"

RUN mkdir -p "$APP_HOME/"

COPY ./$APP_NAME.jar "$APP_HOME/$APP_NAME.jar"
COPY ./lib "$APP_HOME/lib"

ENTRYPOINT ["/opt/apache/beam/boot"]

Thanks for you help.

Deploy Scio jobs with custom containers

0 Answers0