1

I am getting this error while reading data from Kinesis Streams (4 shards) with Flink. Apparently by default, Flink subscribes itself to topic every 5mins and I set parallelism to 3. I also checked this metric Read throughput exceeded - average (Percent) for the Kinesis Stream but seems normal to me. Any clues about it?

java.lang.RuntimeException: Maximum retries exceeded for SubscribeToShard. Failed 10 times. at org.apache.flink.streaming.connectors.kinesis.internals.publisher.fanout.FanOutRecordPublisher.runWithBackoff at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.flink.kinesis.shaded.io.netty.channel.StacklessClosedChannelException at org.apache.flink.kinesis.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source)

I have been searching on Web for topics that mention this error. No luck so far.

Jorge Gomes
  • 11
  • 1
  • Aren't there any more logs suggesting what went wrong? `ChannelClosedException` means that the channel has been closed for some reason, so there should be some info regarding why this happened. – Dominik Wosiński Jan 24 '23 at 16:10

0 Answers0