0

I am running an ExecutorService of more than 50 threads concurrently. Each thread is opening a connection to Cassandra and performing inserts using springframework.data.cassandra. The problem is when I open more than 50 connections at a time, I get the following error.

Caused by: org.jboss.netty.channel.ChannelException: Failed to create a selector.
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:343)
    at org.jboss.netty.channel.socket.nio.AbstractNioSelector.<init>(AbstractNioSelector.java:100)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:52)
    at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)
    at org.jboss.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:45)
    at org.jboss.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:28)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.newWorker(AbstractNioWorkerPool.java:143)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:81)
    at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)
    at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)
    at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.<init>(NioClientSocketChannelFactory.java:151)
    at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.<init>(NioClientSocketChannelFactory.java:116)
    at com.datastax.driver.core.Connection$Factory.<init>(Connection.java:532)
    at com.datastax.driver.core.Cluster$Manager.<init>(Cluster.java:1201)
    at com.datastax.driver.core.Cluster$Manager.<init>(Cluster.java:1144)
    at com.datastax.driver.core.Cluster.<init>(Cluster.java:121)
    at com.datastax.driver.core.Cluster.<init>(Cluster.java:108)
    at com.datastax.driver.core.Cluster.buildFrom(Cluster.java:177)
    at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:1109)

If I open exactly 50 threads (or less), it works fine. Is there a way to configure this so I can allow more? In my cassandra.yaml file, rpc_max_threads according to the comments by default "The default is unlimited"

Chris Lohfink
  • 16,150
  • 1
  • 29
  • 38
user1738539
  • 874
  • 2
  • 11
  • 23

1 Answers1

1

My guess is you are overwhelming your OS by creating too many connections. You should only create 1 Cluster instance per Cassandra cluster. Clusters create Sessions, which manage their own connection pools. Both Cluster and Session are thread safe, so you can share them between threads.

Four simple rules for coding with the driver distills these concepts well:

When writing code that uses the driver, there are four simple rules that you should follow that will also make your code efficient:

  1. Use one cluster instance per (physical) cluster (per application lifetime)
  2. Use at most one session instance per keyspace, or use a single Session and explicitly specify the keyspace in your queries ...

A Cluster instance allows to configure different important aspects of the way connections and queries will be handled. At this level you can configure everything from contact points (address of the nodes to be contacted initially before the driver performs node discovery), the request routing policy, retry and reconnection policies, and so forth. Generally such settings are set once at the application level.

While the session instance is centered around query execution, the Session it also manages the per-node connection pools. The session instance is a long-lived object, and it should not be used in a request-response, short-lived fashion. The code should share the same cluster and session instances across your application.

Community
  • 1
  • 1
Andy Tolbert
  • 11,418
  • 1
  • 30
  • 45