Im having trouble using Thingsboard platform (IoT) when simulating 7.5K devices sending data to the platform. I have the following error in the logs as soon as I start sending data (over MQTT):
2020-08-01 01:17:06,946 [ForkJoinPool-12-worker-0] ERROR c.g.c.u.concurrent.AggregateFuture - Got more than one input Future failure. Logging failures after the first
java.lang.IllegalStateException: Deque full
at java.util.concurrent.LinkedBlockingDeque.addLast(LinkedBlockingDeque.java:335)
at java.util.concurrent.LinkedBlockingDeque.add(LinkedBlockingDeque.java:633)
at org.thingsboard.server.dao.util.AbstractBufferedRateExecutor.submit(AbstractBufferedRateExecutor.java:109)
at org.thingsboard.server.dao.nosql.CassandraAbstractDao.executeAsync(CassandraAbstractDao.java:93)
at org.thingsboard.server.dao.nosql.CassandraAbstractDao.executeAsyncWrite(CassandraAbstractDao.java:76)
at org.thingsboard.server.dao.timeseries.CassandraBaseTimeseriesDao.savePartition(CassandraBaseTimeseriesDao.java:434)
at org.thingsboard.server.dao.timeseries.BaseTimeseriesService.saveAndRegisterFutures(BaseTimeseriesService.java:153)
at org.thingsboard.server.dao.timeseries.BaseTimeseriesService.save(BaseTimeseriesService.java:144)
at org.thingsboard.server.service.telemetry.DefaultTelemetrySubscriptionService.saveAndNotify(DefaultTelemetrySubscriptionService.java:124)
at org.thingsboard.rule.engine.telemetry.TbMsgTimeseriesNode.onMsg(TbMsgTimeseriesNode.java:89)
at org.thingsboard.server.actors.ruleChain.RuleNodeActorMessageProcessor.onRuleChainToRuleNodeMsg(RuleNodeActorMessageProcessor.java:107)
at org.thingsboard.server.actors.ruleChain.RuleNodeActor.onRuleChainToRuleNodeMsg(RuleNodeActor.java:97)
at org.thingsboard.server.actors.ruleChain.RuleNodeActor.doProcess(RuleNodeActor.java:60)
at org.thingsboard.server.actors.service.ContextAwareActor.process(ContextAwareActor.java:45)
at org.thingsboard.server.actors.TbActorMailbox.processMailbox(TbActorMailbox.java:121)
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
I have try to google and see the reason behind it, but i havent found anything.
While simulating with 5K devices, this error came like 3 times each day (over a 4 day period), but it stopped showing that error eventually. However, when increasing the number of devices, the error is constant. Im using Kafka as the broker, but I dont see any Kafka related error. I just want to know why the error appears, is it related to memory, or any other limit?
Thanks in advance
Francisco P
