when we send lots of request to Cassandra we are getting sometimes
ERROR [ReadRepairStage:12] 2018-05-07 10:35:19,056 CassandraDaemon.java:228 - Exception in thread Thread[ReadRepairStage:12,5,main]
org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.
at org.apache.cassandra.service.DataResolver$RepairMergeListener.close(DataResolver.java:199) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.close(UnfilteredPartitionIterators.java:175) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:92) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.service.DataResolver.compareResponses(DataResolver.java:76) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.service.AsyncRepairCallback$1.runMayThrow(AsyncRepairCallback.java:50) ~[apache-cassandra-3.11.1.jar:3.11.1]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.11.1.jar:3.11.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.8.0_152]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.8.0_152]
at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81) ~[apache-cassandra-3.11.1.jar:3.11.1]
at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_152]
any idea what causing this? any missing information i can give?
I also see
INFO [ReadStage-14] 2018-05-07 10:22:58,869 NoSpamLogger.java:91 - Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB
I suspect that might help to cause the problem? No idea.
Update: we have multi node with multi instance (6 node's each node is 4 instance's) also we tried to tune GC as we suspect that cause slowness
