Periodically receive "InvalidQueryException: unconfigured columnfamily documents"

Question

We have an app that is basically polling periodically for changes every 1 minute, using Datastax c* driver 2.0.4 (also tried 2.1.0-beta1).

We create a Session at app startup, specifying the keyspace, e.g. Session session = cluster.connect("my_keyspace");

Then using a Timer execute the following selects every minute:

Statement stmt = QueryBuilder.select().column("column2").from("my_keyspace","document_dates").where(QueryBuilder.in("key", date_keys));
ResultSet results = session.execute(stmt);

and later:

stmt = QueryBuilder.select().from("my_keyspace","documents").where(QueryBuilder.in("key", rows.toArray(new Object[rows.size()])));
results = session.execute(stmt);

This seems to work fine, except that every 6 minutes I see an exception indicating "unconfigured columnfamily documents"

com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured columnfamily documents
    at com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:35)
    at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:256)
    at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:172)
    at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
    at com.mine.MyClass$1.run(AutoModInserter.java:129)
    at java.util.TimerThread.mainLoop(Timer.java:555)
    at java.util.TimerThread.run(Timer.java:505)
Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured columnfamily documents
    at com.datastax.driver.core.Responses$Error.asException(Responses.java:97)
    at com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:108)
    at com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:235)
    at com.datastax.driver.core.RequestHandler.onSet(RequestHandler.java:367)
    at com.datastax.driver.core.Connection$Dispatcher.messageReceived(Connection.java:584)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)

I have seen a few references to keyspaces with regards to incorrectly specifying them in different clients, or using upper case chars requiring quotes, etc, but nothing quite like what we are experiencing. I'm thrown by the frequency of this exception not matching the frequency of the timer, but still being predictable in happening every 6th iteration.

The closest thing I have found is this unanswered question: unconfigured columnfamily error on consecutive execute calls (CQL)

Thanks for any advice

Is it possible one or more of your nodes has a schema disagreement and you're just getting around to it every 6 minutes? You might try "nodetool gossipinfo" and see if they all have the same. — Adam Holmberg, Aug 19 '14 at 13:40
Also, in your application, you might log the coordinator node from the result execution info and see if you can narrow down the node. Then, check server logs for the errant node and see if anything pops out. — Adam Holmberg, Aug 19 '14 at 13:41

Periodically receive "InvalidQueryException: unconfigured columnfamily documents"

0 Answers0