1

I have a fat jar called producer which produces messages.I want to use it to produce messages to a MSK serverless cluster. The jar takes the following arguments-

-topic --num-records  --record-size  --throughput --producer.config /configLocation/

As my MSK serverless cluster uses IAM based authentication, i have provided the following settings in my producer.config-

bootstrap.servers=boot-url
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

The way this jar usually works is by providing a username and password with the sasl.jaas.config property.

However, with MSK serverless we have to use the IAM role attached to our EC2 instance.

When my executing the current jar by using

java - jar producer.jar -topic --num-records  --record-size  --throughput --producer.config /configLocation/

I get the exception

Exception in thread "main" org.apache.kafka.common.config.ConfigException: Invalid value software.amazon.msk.auth.iam.IAMClientCallbackHandler for configuration sasl.client.callback.handler.class: Class software.amazon.msk.auth.iam.IAMClientCallbackHandler could not be found.

I don't understand how to make the producer jar find the class present in the external jar aws-msk-iam-auth-1.1.1-all.jar.

Any help would be much appreciated, Thanks.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Gestalt
  • 93
  • 10
  • What are you using to create your jar? Is it an uber jar? Did you add the IAM library as a dependency? – OneCricketeer Oct 11 '22 at 14:58
  • The fat/uber jar is a blackbox but I'm pretty sure they wouldn't have a dependency on anything aws related as they are availing the services of a different cloud provider. – Gestalt Oct 11 '22 at 15:05
  • 1
    You can use `jar -tf` to examine that "black box", but as the error says, that class is not available on the classpath... You can `export CLASSPATH` or use `-cp` flag of `java` command to otherwise include external JARs in your runtime execution. – OneCricketeer Oct 11 '22 at 17:51
  • Thanks Mr. Cricketeer. I examined the jar and confirmed the dependency isn't present . I'm gonna be using the explicitly specified classpath in the meantime. – Gestalt Oct 12 '22 at 03:49
  • If that works for you, feel free to provide the full command in an answer below – OneCricketeer Oct 12 '22 at 13:18
  • 1
    I didn't get the chance to try manually setting the classpath as we were able to request the jar to be updated with required AWS IAM dependencies. TPTB didn't want to modify the cp for this singular jar when the option of inclusion was available. – Gestalt Oct 17 '22 at 06:48
  • @OneCricketeer I have answered the question based on my findings. – Gestalt Oct 28 '22 at 15:05

1 Answers1

0

I found out that it isn't possible to override the classpath specified in MANIFEST.MF in case of a jar by using commandline options like --cp. What worked in my case was to have the jar's pom include the missing dependency.

When you use the -jar command-line option to run your program as an executable JAR, then the Java CLASSPATH environment variable will be ignored, and also the -cp and -classpath switches will be ignored and, In this case, you can set your java classpath in the META-INF/MANIFEST.MF file by using the Class-Path attribute.

https://javarevisited.blogspot.com/2011/01/how-classpath-work-in-java.html

Gestalt
  • 93
  • 10