Distributed Cache - Terracotta & Ehcache - Client error: Unable to create toolkit

Question

I am implementing a distributed cache using ehcache (2.10.1) and terracotta (open source 4.3.1). I tried with two JVM instances and a terracotta server on a single machine and the code worked as expected without errors.

Now I am trying to run the same terracotta server but with the client on a virtual machine. When I run the client on the VM, I get the following messages and errors:

2016-02-23 14:27:10,943  INFO TerracottaClient:209 - Thread [main] [cacheManager: no name]: Creating new ClusteredInstanceFactory
2016-02-23 14:27:10,954  WARN TerracottaClusteredInstanceHelper:148 - ehcache-terracotta jar is detected in the current classpath. The use of ehcache-terracotta jar is no longer needed in this version of Ehcache.

2016-02-23 14:27:11,241 INFO - Terracotta 4.3.1, as of 20151016-121959 (Revision 26592 from 4.3.1)
2016-02-23 14:27:12,177 INFO - Successfully loaded base configuration from server at 'A.B.C.D:9510'.

2016-02-23 14:27:12,222 INFO - Log file: '/root/terracotta/client-logs/terracotta-client.log'.

2016-02-23 14:27:18,202 INFO - Connection successfully established to server at A.B.C.D:9510
2016-02-23 14:27:24,731  INFO AnnotationSizeOfFilter:53 - Using regular expression provided through VM argument net.sf.ehcache.pool.sizeof.ignore.pattern for IgnoreSizeOf annotation : ^.*cache\..*IgnoreSizeOf$

Exception in thread "main" net.sf.ehcache.CacheException: Unable to load class net.sf.ehcache.terracotta.StandaloneTerracottaClusteredInstanceFactory. Initial cause was org.terracotta.toolkit.ToolkitInstantiationException: java.lang.RuntimeException: Unable to create toolkit.
    at net.sf.ehcache.util.ClassLoaderUtil.createNewInstance(ClassLoaderUtil.java:90)
    at net.sf.ehcache.terracotta.TerracottaClusteredInstanceHelper.newClusteredInstanceFactory(TerracottaClusteredInstanceHelper.java:157)
    at net.sf.ehcache.terracotta.TerracottaClient.createNewClusteredInstanceFactory(TerracottaClient.java:180)
    at net.sf.ehcache.terracotta.TerracottaClient.createClusteredInstanceFactory(TerracottaClient.java:129)
    at net.sf.ehcache.CacheManager.doInit(CacheManager.java:463)
    at net.sf.ehcache.CacheManager.init(CacheManager.java:395)
    at net.sf.ehcache.CacheManager.<init>(CacheManager.java:295)
    at com.cisco.dcextract.ExtractCache.main(ExtractCache.java:13)

The tc-config.xml file used:

<?xml version="1.0" encoding="UTF-8"?>
<con:tc-config xmlns:con="http://www.terracotta.org/config">
  <servers>
    <server host="A.B.C.D" name="tcservername" >
      <data>c:\datacache</data>
      <tsa-port>9510</tsa-port>
      <jmx-port>9520</jmx-port>
    <tsa-group-port>9530</tsa-group-port>
    <offheap>
              <enabled>true</enabled>
              <maxDataSize>2g</maxDataSize>
     </offheap>
      <logs>terracotta/server-logs</logs>
     </server>
    <update-check>
      <enabled>true</enabled>
    </update-check>
  </servers>
  <clients>
    <logs>terracotta/client-logs</logs>
  </clients>  
</con:tc-config>

The ehcache.xml file used:

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="ehcache.xsd"
             updateCheck="true" monitoring="autodetect">
             <terracottaConfig url="A.B.C.D:9510" />

   <defaultCache
            maxElementsInMemory="10000"
            eternal="false"
            timeToIdleSeconds="120"
            timeToLiveSeconds="120"
            overflowToDisk="true"
            diskSpoolBufferSizeMB="30"
            diskPersistent="false"
            diskExpiryThreadIntervalSeconds="120"
            memoryStoreEvictionPolicy="LRU"
            />

    <cache name="cache1290"
           maxElementsInMemory="1000000"
           eternal="false"
           overflowToDisk="false"
           diskSpoolBufferSizeMB="20"
           timeToIdleSeconds="100"
           timeToLiveSeconds="100"
           memoryStoreEvictionPolicy="LRU">
    <terracotta/> 
     </cache>
</ehcache>

While running the program on the client, I had included the terracotta-toolkit-runtime-4.3.1.jar & other required jars. I am trying to debug the error. What could be the possible reason for the error?

Log file content(Adding only relevant part)

2016-02-25 15:37:28,852 [main] INFO com.terracottatech.console - Connection successfully established to server at 173.39.65.73:9510

2016-02-25 15:37:28,852 [WorkerThread(client_coordination_stage, 0)] INFO com.tc.object.RemoteServerMapManager - ClientID[6]: Sending outstanding servermap requests, num msgs: 0

2016-02-25 15:37:28,853 [main] INFO com.terracottatech.tsa - Connection successfully established to server at 173.39.65.73:9510

2016-02-25 15:37:28,853 [WorkerThread(client_coordination_stage, 0)] INFO com.tc.management.remote.protocol.terracotta.TunnelingEventHandler - Client JMX server ready; sending notification to L2 server

2016-02-25 15:37:28,853 [WorkerThread(client_coordination_stage, 0)] INFO com.tc.platform.rejoin.RejoinManagerImpl - This node joined the cluster - rejoinEnabled: false newNodeId: ClientID[6]

2016-02-25 15:37:28,855 [WorkerThread(client_coordination_stage, 0)] INFO com.tc.cluster.DsoClusterImpl - NODE_JOINED ClientID[6] rejoinHappened false

2016-02-25 15:37:33,417 [main] INFO com.tc.management.remote.protocol.terracotta.TunneledDomainManager - Sending current registered tunneled domains to L2 server to set up the tunneled connections for the mbeans that match.

2016-02-25 15:37:34,553 [L1 VM Shutdown Hook] INFO com.terracottatech.tsa - Running L1 VM shutdown hook

2016-02-25 15:37:34,553 [L1 VM Shutdown Hook] INFO com.terracottatech.tsa - shuting down Terracotta Client hook=true force=false

2016-02-25 15:37:34,553 [L1 VM Shutdown Hook] INFO com.terracotta.toolkit.factory.impl.ToolkitNotifierFactoryImpl - Shutting Down Notifier Thread Pool
2016-02-25 15:37:34,554 [CommonShutDownHook] INFO com.terracottatech.tsa - L1 Exiting...
2016-02-25 15:37:34,789 [L1 VM Shutdown Hook] INFO com.tc.object.tx.RemoteTransactionManagerImpl - ClientID[6]: stop(): took 0 millis to complete
2016-02-25 15:37:34,790 [L1 VM Shutdown Hook] INFO com.tc.net.protocol.transport.ConnectionHealthCheckerImpl: DSO Client - Connection to [dhcp-173-39-65-73.cisco.com:9510] CLOSED. Health Monitoring for this node is now disabled.
2016-02-25 15:37:34,790 [L1 VM Shutdown Hook] INFO com.tc.net.protocol.transport.ClientConnectionEstablisher - waiting for connection establisher to finish null
2016-02-25 15:37:34,790 [L1 VM Shutdown Hook] INFO com.tc.platform.rejoin.ClientChannelEventController - Got channel event - type: CHANNEL_CLOSED_EVENT, event: com.tc.net.protocol.tcm.ChannelEventImpl@1768481525[type = CHANNEL_CLOSED_EVENT, timestamp = Thu Feb 25 15:37:34 IST 2016, channel  = ChannelID[NULL_ID, Status:CLOSED]:10.197.70.109:35582 <--> 173.39.65.73:9510 remote node  : GroupID[0]

Anything in the `/root/terracotta/client-logs/terracotta-client.log` log file? Did you try chasing down and removing `ehcache-terracotta.jar` from your classpath? — Louis Jacomet, Feb 26 '16 at 09:50
After removing ehcache-terracotta jar file, I am not getting the warning message but still getting the same errors. I added the client-log file — RohitChaware, Feb 26 '16 at 11:28
So the connection to the Terracotta server is established and something else crashes locally it seems. Could there be another exception in the logs? Is there more to the stack trace that you did not share? — Louis Jacomet, Feb 26 '16 at 23:12

score 0 · Answer 1 · answered Feb 26 '19 at 12:30

The issue should be due to missed dependency or version mismatch. I too faced the same issue, solved it by adding below combination of dependencies.

<dependencies>
    <dependency>
        <groupId>net.sf.ehcache</groupId>
        <artifactId>ehcache-core</artifactId>
        <version>2.5.6</version>
    </dependency>
    <dependency>
        <groupId>net.sf.ehcache</groupId>
        <artifactId>ehcache-terracotta</artifactId>
        <version>2.5.6</version>
    </dependency>
    <dependency>
        <groupId>org.terracotta</groupId>
        <artifactId>terracotta-toolkit-1.5-runtime</artifactId>
        <version>4.5.0</version>
    </dependency>
    <dependency>
        <groupId>org.terracotta</groupId>
        <artifactId>terracotta-toolkit-runtime</artifactId>
        <version>4.3.2</version>
    </dependency>
</dependencies>

<repositories>
    <repository>
        <id>terracotta-repository</id>
        <url>http://www.terracotta.org/download/reflector/releases</url>
        <releases>
            <enabled>true</enabled>
        </releases>
    </repository>
</repositories>

score 0 · Answer 2 · answered Oct 19 '19 at 03:08

I had the same problem, It got fixed by adding:

    <dependency>
        <groupId>net.sf.ehcache</groupId>
        <artifactId>ehcache-terracotta</artifactId>
        <version>2.5.6</version>
    </dependency>
    <dependency>
        <groupId>org.terracotta</groupId>
        <artifactId>terracotta-toolkit-runtime-ee</artifactId>
        <version>4.1.1</version>
    </dependency>

and the repo:

<repositories>
    <repository>
        <id>terracotta-repository</id>
        <url>http://www.terracotta.org/download/reflector/releases</url>
        <releases>
            <enabled>true</enabled>
        </releases>
    </repository>
</repositories>

Distributed Cache - Terracotta & Ehcache - Client error: Unable to create toolkit

2 Answers2