Too many open files erroe in cassandra driver

Question

I am using DataStax Java Driver 2.17 and I am getting this error:

Caused by: io.netty.channel.ChannelException: Unable to create Channel from class class io.netty.channel.socket.nio.NioSocketChannel
    at io.netty.bootstrap.AbstractBootstrap$BootstrapChannelFactory.newChannel(AbstractBootstrap.java:455) ~[netty-transport-4.0.27.Final.jar:4.0.27.Final]
    at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:306) ~[netty-transport-4.0.27.Final.jar:4.0.27.Final]
    at io.netty.bootstrap.Bootstrap.doConnect(Bootstrap.java:133) ~[netty-transport-4.0.27.Final.jar:4.0.27.Final]
    at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:115) ~[netty-transport-4.0.27.Final.jar:4.0.27.Final]
    at com.datastax.driver.core.Connection.initAsync(Connection.java:136) ~[cassandra-driver-core-2.1.7.jar:na]
    at com.datastax.driver.core.HostConnectionPool.initAsync(HostConnectionPool.java:119) ~[cassandra-driver-core-2.1.7.jar:na]
    at com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:265) ~[cassandra-driver-core-2.1.7.jar:na]
    at com.datastax.driver.core.SessionManager.maybeAddPool(SessionManager.java:301) ~[cassandra-driver-core-2.1.7.jar:na]
    at com.datastax.driver.core.SessionManager.createPoolsInParallel(SessionManager.java:81) ~[cassandra-driver-core-2.1.7.jar:na]
    at com.datastax.driver.core.SessionManager.init(SessionManager.java:70) ~[cassandra-driver-core-2.1.7.jar:na]
    at com.datastax.driver.core.Cluster.connect(Cluster.java:251) ~[cassandra-driver-core-2.1.7.jar:na]
    at com.symantec.nsp.cassandra.CassandraConnector.getSession(CassandraConnector.java:80) ~[cassandra-client-1.0.1-SNAPSHOT.jar:na]
    at com.symantec.nsp.storageservice.dao.impl.cassandra.BucketDAOImpl.upsert(BucketDAOImpl.java:45) ~[storageservice-cassandra-1.0.0-SNAPSHOT.jar:na]
    at com.symantec.nsp.storageservice.dao.impl.cassandra.BucketDAOImpl.create(BucketDAOImpl.java:34) ~[storageservice-cassandra-1.0.0-SNAPSHOT.jar:na]
    at com.symantec.nsp.storageservice.service.impl.BucketServiceImpl.createBucket(BucketServiceImpl.java:79) ~[BucketServiceImpl.class:na]
    ... 76 common frames omitted
Caused by: io.netty.channel.ChannelException: Failed to open a socket.
    at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:62) ~[netty-transport-4.0.27.Final.jar:4.0.27.Final]
    at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:72) ~[netty-transport-4.0.27.Final.jar:4.0.27.Final]
    at sun.reflect.GeneratedConstructorAccessor42.newInstance(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_20]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408) ~[na:1.8.0_20]
    at java.lang.Class.newInstance(Class.java:433) ~[na:1.8.0_20]
    at io.netty.bootstrap.AbstractBootstrap$BootstrapChannelFactory.newChannel(AbstractBootstrap.java:453) ~[netty-transport-4.0.27.Final.jar:4.0.27.Final]
    ... 90 common frames omitted
Caused by: java.net.SocketException: Too many open files
    at sun.nio.ch.Net.socket0(Native Method) ~[na:1.8.0_20]
    at sun.nio.ch.Net.socket(Net.java:415) ~[na:1.8.0_20]
    at sun.nio.ch.Net.socket(Net.java:408) ~[na:1.8.0_20]
    at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:105) ~[na:1.8.0_20]
    at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60) ~[na:1.8.0_20]
    at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:60) ~[netty-transport-4.0.27.Final.jar:4.0.27.Final]

Any ideas what to tweak to fix this?

Answer 1

Please make sure that your system meets the recommended production settings. You'll likely have to increase the max. number of open files using the nofile setting in limits.conf.

Answer 2

Fixed this by creating only one instance of Session. Earlier session was getting created per request.