1

I've built a Quarkus 2.7.1 console application using picocli that includes several subcommands. I'd like to be able to run this application within a Kubernetes cluster and decide its arguments at run-time. This is so that I can use the same container image to run the application in different modes within the cluster.

To get things started I added the JIB extension and tried setting the arguments using a configuration value quarkus.jib.jvm-arguments. Unfortunately it seems like this configuration value is locked at build-time so I'm unable to update this at run-time.

Next I tried setting quarkus.args while using default settings for JIB. The configuration value documentation makes it sound general enough for the job but it doesn't seem to have an affect when the application is run in the container. Since most references to this configuration value in documentation are in the context of Dev Mode I'm wondering if this may be disabled outside of that.

How can I get this application running in a container image with its arguments decided at run-time?

oaklandcorp-jkaiser
  • 128
  • 1
  • 6
  • What sort of configuration are you trying to set? And how do you plan to obtain the values in Kubernetes? – geoand Feb 15 '22 at 06:45
  • So far I've used environment variables as my key mechanism for toggling behavior in the Quarkus container images I've built. From my naive perspective I was hoping to set an environment variable in Kubernetes to the arguments string (ex: `initialize --no-timeout`) which would be used by the internal application on startup. – oaklandcorp-jkaiser Feb 15 '22 at 14:34
  • Environment variables are not set in the run script in Kubernetes. There is dedicated section in the Pod – geoand Feb 15 '22 at 14:55
  • To be more specific the cluster I'm trying to run this container in is managed by a series of Helmfiles so my perspective of the problem may be influenced by that. My experimentation so far would include adding a new chart whose definition includes my container image and a set of environment variables. Behind the scenes it must handle the creation of the pod with the environment variables passed through from the chart. – oaklandcorp-jkaiser Feb 15 '22 at 15:16
  • Speaking practically I believe I can achieve what I need in the cluster by other means (building a container image per Kubernetes use-case with built-time locked arguments or maybe a single custom `Dockerfile` with an entrypoint that tries to pull arguments from an environment variable) but I wanted to ask the community first to see if there was a better way to do it. – oaklandcorp-jkaiser Feb 15 '22 at 15:57

3 Answers3

2

You can set quarkus.jib.jvm-entrypoint to any container entrypoint command you want, including scripts. An example in the doc is quarkus.jib.jvm-entrypoint=/deployments/run-java.sh. You could make use of $CLI_ARGUMENTS in such a script. Even something like quarkus.jib.jvm-entrypoint=/bin/sh,-c,'/deployments/run-java.sh $CLI_ARGUMENTS' should work too, as long as you place the script run-java.sh at /deployments in the image. The possibility is limitless.

Also see this SO answer if there's an issue. (The OP in the link put a customer script at src/main/jib/docker/run-java.sh (src/main/jib is Jib's default "extra files directory") so that Jib places the script in the image at /docker/run-java.sh.

Chanseok Oh
  • 3,920
  • 4
  • 23
  • 63
  • I've tried the suggested `quarkus.jib.jvm-entrypoint=/bin/sh,-c,'/deployments/run-java.sh $CLI_ARGUMENTS'` value combined with the `quarkus.jib.environment-variables."JAVA_APP_DIR"=/work ` value mentioned in the documentation and receive the error message `/bin/sh: /deployments/run-java.sh $CLI_ARGUMENTS: No such file or directory` when trying to start up the container in the cluster. I've even tried trimming it back to `quarkus.jib.jvm-entrypoint=/bin/sh,/deployments/run-java.sh` and noticed the same error message. Any suggestions for troubleshooting? – oaklandcorp-jkaiser Feb 18 '22 at 21:10
  • Hmm, I wonder if it might have to do with the default JVM image used. [Here](https://quarkus.io/guides/container-image#jvm-debugging) documentation suggests `fabric8/java-alpine-openjdk11-jre` is the default image used but [here](https://quarkus.io/guides/container-image#quarkus-container-image-jib_configuration) seems to suggest `registry.access.redhat.com/ubi8/openjdk-11-runtime:1.11` is the default. – oaklandcorp-jkaiser Feb 18 '22 at 22:03
  • Alright, after a bunch of experimenting this morning I think I found the winning combo. Using the default JVM image (I believe that is `registry.access.redhat.com/ubi8/openjdk-11-runtime`) and the entrypoint `quarkus.jib.jvm-entrypoint=/bin/sh,-c,java -jar /work/quarkus-run.jar $CLI_ARGUMENTS` I'm able to run my CLI with program arguments coming from an environment variable. This solution seems to behave the same as my custom `Dockerfile` solution, but has the added bonus of not needing to maintain a custom `Dockerfile`. Very nice, thank you! – oaklandcorp-jkaiser Feb 21 '22 at 16:10
  • Just as a follow-up, this issue tracks the differing documentation: https://github.com/quarkusio/quarkus/issues/23841 – oaklandcorp-jkaiser Feb 22 '22 at 13:45
0

I was able to find a solution to the problem with a bit of experimenting this morning.

With the quarkus-container-image-docker extension (instead of quarkus.jib.jvm-arguments) I was able to take the template Dockerfile.jvm and extend it to pass through arguments to the CLI. The only line that needed changing was the ENTRYPOINT (details included in the snippet below). I changed the ENTRYPOINT form (from exec to shell) and added an environment variable as an argument to pass-through program arguments.

FROM registry.access.redhat.com/ubi8/ubi-minimal:8.3

ARG JAVA_PACKAGE=java-11-openjdk-headless
ARG RUN_JAVA_VERSION=1.3.8
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en'
# Install java and the run-java script
# Also set up permissions for user `1001`
RUN microdnf install curl ca-certificates ${JAVA_PACKAGE} \
    && microdnf update \
    && microdnf clean all \
    && mkdir /deployments \
    && chown 1001 /deployments \
    && chmod "g+rwX" /deployments \
    && chown 1001:root /deployments \
    && curl https://repo1.maven.org/maven2/io/fabric8/run-java-sh/${RUN_JAVA_VERSION}/run-java-sh-${RUN_JAVA_VERSION}-sh.sh -o /deployments/run-java.sh \
    && chown 1001 /deployments/run-java.sh \
    && chmod 540 /deployments/run-java.sh \
    && echo "securerandom.source=file:/dev/urandom" >> /etc/alternatives/jre/lib/security/java.security

# Configure the JAVA_OPTIONS, you can add -XshowSettings:vm to also display the heap size.
ENV JAVA_OPTIONS="-Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
# We make four distinct layers so if there are application changes the library layers can be re-used
COPY --chown=1001 target/quarkus-app/lib/ /deployments/lib/
COPY --chown=1001 target/quarkus-app/*.jar /deployments/
COPY --chown=1001 target/quarkus-app/app/ /deployments/app/
COPY --chown=1001 target/quarkus-app/quarkus/ /deployments/quarkus/

EXPOSE 8080
USER 1001

# [== BEFORE ==]
# ENTRYPOINT [ "/deployments/run-java.sh" ]
# [== AFTER ==]
ENTRYPOINT "/deployments/run-java.sh" $CLI_ARGUMENTS
oaklandcorp-jkaiser
  • 128
  • 1
  • 6
0

I have tried the above approaches but they didn't work with the default quarkus JIB's ubi8/openjdk-17-runtime image. This is because this base image doesn't use /work as the WORKIR, but instead the /home/jboss.

Therefore, I created a custom start-up script and referenced it on the properties file as following. This approach works better if there's a need to set application params using environment variables:

File: application.properties

quarkus.jib.jvm-entrypoint=/bin/sh,run-java.sh

File: src/main/jib/home/jboss/run-java.sh

java \
  -Djavax.net.ssl.trustStore=/deployments/truststore \
  -Djavax.net.ssl.trustStorePassword="$TRUST_STORE_PASSWORD" \
  -jar quarkus-run.jar
João Pedro Schmitt
  • 1,046
  • 1
  • 11
  • 25