We use spymemcached 2.11.4 library and have been noticing a lot of warning messages (disconnect/reconnect) in our logs, our clients disconnects and reconnects very often (roughly once a minute)
Does anyone know why these can happen:
2014-09-29 12:58:21.186 INFO net.spy.memcached.MemcachedConnection: Reconnecting due to exception on {QA sa=cache.internal/10.212.1.222:11230, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: [115, 116, 97, 116, 115, 13, 10], topWop=null, toWrite=0, interested=1}
java.io.IOException: Disconnected unexpected, will reconnect.
at net.spy.memcached.MemcachedConnection.handleReadsWhenChannelEndOfStream(MemcachedConnection.java:911)
at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:823)
at net.spy.memcached.MemcachedConnection.handleReadsAndWrites(MemcachedConnection.java:720)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:683)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:436)
at net.spy.memcached.MemcachedConnection.run(MemcachedConnection.java:1445)
2014-09-29 12:58:21.187 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=cache.internal/10.212.1.222:11230, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: [115, 116, 97, 116, 115, 13, 10], topWop=null, toWrite=0, interested=1}, attempt 0.
2014-09-29 12:58:21.187 WARN net.spy.memcached.protocol.ascii.AsciiMemcachedNodeImpl: Discarding partially completed op: Cmd: [115, 116, 97, 116, 115, 13, 10]
2014-09-29 12:58:21.187 WARN net.spy.memcached.MemcachedClient: Unsuccessful stat fetch: {OperationStatus success=false: cancelled}
2014-09-29 12:58:21.187 WARN net.spy.memcached.MemcachedClient: Unsuccessful stat fetch: {OperationStatus success=false: cancelled}
2014-09-29 12:58:21.764 INFO net.spy.memcached.MemcachedConnection: Shut down memcached client
2014-09-29 12:59:15.141 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=cache.internal/10.212.1.222:11230, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-09-29 12:59:15.148 INFO net.spy.memcached.MemcachedConnection: Reconnecting due to exception on {QA sa=cache.internal/10.212.1.222:11230, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: [115, 116, 97, 116, 115, 13, 10], topWop=null, toWrite=0, interested=1}
java.io.IOException: Disconnected unexpected, will reconnect.
at net.spy.memcached.MemcachedConnection.handleReadsWhenChannelEndOfStream(MemcachedConnection.java:911)
at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:823)
at net.spy.memcached.MemcachedConnection.handleReadsAndWrites(MemcachedConnection.java:720)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:683)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:436)
at net.spy.memcached.MemcachedConnection.run(MemcachedConnection.java:1445)
2014-09-29 12:59:15.149 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=cache.internal/10.212.1.222:11230, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: [115, 116, 97, 116, 115, 13, 10], topWop=null, toWrite=0, interested=1}, attempt 0.
2014-09-29 12:59:15.149 WARN net.spy.memcached.protocol.ascii.AsciiMemcachedNodeImpl: Discarding partially completed op: Cmd: [115, 116, 97, 116, 115, 13, 10]
2014-09-29 12:59:15.149 WARN net.spy.memcached.MemcachedClient: Unsuccessful stat fetch: {OperationStatus success=false: cancelled}
2014-09-29 12:59:15.149 WARN net.spy.memcached.MemcachedClient: Unsuccessful stat fetch: {OperationStatus success=false: cancelled}
2014-09-29 12:59:16.726 INFO net.spy.memcached.MemcachedConnection: Shut down memcached client
2014-09-29 13:00:31.360 INFO net.spy.memcached.MemcachedConnection: Added {QA sa=cache.internal/10.212.1.222:11230, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-09-29 13:00:31.368 INFO net.spy.memcached.MemcachedConnection: Reconnecting due to exception on {QA sa=cache.internal/10.212.1.222:11230, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: [115, 116, 97, 116, 115, 13, 10], topWop=null, toWrite=0, interested=1}
java.io.IOException: Disconnected unexpected, will reconnect.
at net.spy.memcached.MemcachedConnection.handleReadsWhenChannelEndOfStream(MemcachedConnection.java:911)
at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:823)
at net.spy.memcached.MemcachedConnection.handleReadsAndWrites(MemcachedConnection.java:720)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:683)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:436)
at net.spy.memcached.MemcachedConnection.run(MemcachedConnection.java:1445)
