0

We have a single node Cassandra and time to time we get the following error:

    info_outline   04:32:41.354 octopus-cassandra-standalone /var/log/application.log  
 docker/d9cce2283ebc[902]: INFO  [epollEventLoopGroup-2-8] 2018-10-07 02:32:38,050 Message.java:623 - Unexpected exception during request; channel = [id: 0xdfa793db, L:/172.31.152.43:9042 - R:/172.31.0.170:7154]
info_outline   04:32:41.354 octopus-cassandra-standalone   

/var/log/application.log Oct  7 02:32:38 ip-172-31-152-43 docker/d9cce2283ebc[902]: io.netty.channel.unix.Errors$NativeIoException: readAddress() failed: Connection timed out

More info: Kairosdb gets time out connecting to the cassandra host e.g. "cassandra-host.me" .
I already found that 172.31.152 is the IP address of the Cassandra DNS ("cassandra-host.me"). What is confusing for me is, what is the 172.31.0.170 IP addresses?

The 2nd IP is changing time to time and I could see the service was always running without any problem.

Please let me know if you need more info to help.

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
user1859675
  • 201
  • 2
  • 5

1 Answers1

0

Address that is marked as R is remote address of the client that is connected to Cassandra (L address). This is printed representation of Netty's Channel object.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • Thanks Alex. That means, the client (KairosDB) gets timeout connecting to C* (which is correct). Any idea why time out is happening? – user1859675 Oct 10 '18 at 15:09
  • Node could be overloaded (for example, doing garbage collection), maybe KairosDB has not optimal queries (for example, with `ALLOW FILTERING`) - it's hard to say - you need to check the logs on Cassandra side for that time – Alex Ott Oct 10 '18 at 16:46