0

Flink version: 1.17.1

in local environment through IDE the job is starting and working without any errors,

when starting the job through Flink Web ui in ubuntu environment, the job is started successfully but when any event is pushed to kafka topic (that also updates in UI) but nothing is sink to the result topic and the flink logs shows bellow exception.

org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: org.apache.flink.connector.kafka.sink.KafkaSink
ClassLoader info: URL ClassLoader:
Class not resolvable through given classloader.
        at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:397) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:166) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.<init>(RegularOperatorChain.java:60) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:688) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:675) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:921) [flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) [flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) [flink-dist-1.17.1.jar:1.17.1]
        at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.ClassNotFoundException: org.apache.flink.connector.kafka.sink.KafkaSink
        at java.net.URLClassLoader.findClass(URLClassLoader.java:471) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
        at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:67) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:65) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:51) ~[flink-dist-1.17.1.jar:1.17.1]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
        at org.apache.flink.util.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:192) ~[flink-dist-1.17.1.jar:1.17.1]
        at java.lang.Class.forName0(Native Method) ~[?:?]
        at java.lang.Class.forName(Class.java:398) ~[?:?]
        at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76) ~[flink-dist-1.17.1.jar:1.17.1]
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1995) ~[?:?]
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1862) ~[?:?]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2169) ~[?:?]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679) ~[?:?]
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2464) ~[?:?]
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2358) ~[?:?]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196) ~[?:?]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679) ~[?:?]
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:493) ~[?:?]
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:451) ~[?:?]
        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:534) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:522) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:476) ~[flink-dist-1.17.1.jar:1.17.1]
        at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:383) ~[flink-dist-1.17.1.jar:1.17.1]
        ... 9 more

I tried restarting Flink, inspecting the generated jar if KafkaSink is included or not (which is included) but the result was same, job itself does not show any exception in Web UI nor does it crash, but the flink logs shows that this class is not being found.

bellow is the code used for sink, (this code snippet is part of BroadCastProcessFunction.processElement method)

DataStream<String> dataStream = streamExecutionEnvironment.fromElements(data);

        streamTableEnvironment.createTemporaryView(
                "table",
                dataStream
        );
    private final String sink = "CREATE TABLE KafkaTable (\n" +
            "  `f0` STRING\n" +
            ") WITH (\n" +
            "  'connector' = 'kafka',\n" +
            "  'topic' = 'result',\n" +
            "  'topic-pattern' = 'SINK_TABLE',\n" +
            "  'properties.bootstrap.servers' = 'localhost:9092',\n" +
            "  'properties.group.id' = 'testGroup',\n" +
            "  'value.format' = 'raw'\n" +
            ");";

        // check for null
        Table resultTable = streamTableEnvironment.sqlQuery("SELECT * FROM table");

        streamTableEnvironment.executeSql(sql);

        resultTable.insertInto("KafkaTable").execute();

        streamTableEnvironment.dropTemporaryView(Main.TABLE_NAME);
        streamTableEnvironment.executeSql(drop);
Krutik Gadhiya
  • 1
  • 1

0 Answers0