Distributed cache using infinispan on tomcat fails to start

Question

I'm trying to use infinispan as a distributed cache on my application.

I'm using it as a second level cache for hibernate using spring and JPA.

Infinispan is configured to work in distributed clustered mode using this infinispan-config.xml

<?xml version="1.0" encoding="UTF-8"?>
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"
xmlns="urn:infinispan:config:5.1">

<global>
    <transport clusterName="sigmaCacheCluster">
        <properties>
            <property name="configurationFile" value="jgroups-tcp.xml" />
        </properties>
    </transport>
    <globalJmxStatistics enabled="true" />
</global>
<default>
    <jmxStatistics enabled="true" />
    <clustering mode="replication" >
        <stateRetrieval fetchInMemoryState="true" />
        <sync/>
    </clustering>
</default>
</infinispan>

When using the same configuration file with the demoApp provided with Infinispan it works perfect, how ever when running inside the application and on Tomcat(7) I'm getting the flowing error (Note that the cluster is up and infinispan finds other nodes):

    [DEFAULT APPENDER - FROM WEB] 2013-06-13 11:53:58,248 - INFO org.infinispan.spring.AbstractEmbeddedCacheManagerFactory - Initializing SpringEmbeddedCacheManager instance ...
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:53:58,802 - INFO org.infinispan.configuration.parsing.Parser51 - ISPN000184: The stateRetrieval configuration element has been deprecated, we're assuming you meant stateTransfer. Please see XML schema for more information.
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:53:59,065 - INFO org.infinispan.spring.AbstractEmbeddedCacheManagerFactory - Successfully initialized SpringEmbeddedCacheManager instance [org.infinispan.spring.provider.SpringEmbeddedCacheManager@6ee47e99]
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:53:59,732 - INFO org.infinispan.remoting.transport.jgroups.JGroupsTransport - ISPN000078: Starting JGroups Channel
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:54:03,263 - INFO org.jgroups.protocols.UFC - UFC is not needed (and can be removed) as we're running on a TCP transport
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:54:03,581 - INFO org.infinispan.remoting.transport.jgroups.JGroupsTransport - ISPN000094: Received new cluster view: [comp-Lap-28466|5] [comp-Lap-28466, comp-Lap-1139]
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:54:03,852 - INFO org.infinispan.remoting.transport.jgroups.JGroupsTransport - ISPN000079: Cache local address is comp-Lap-1139, physical addresses are [10.0.0.106:7801]
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:54:03,858 - INFO org.infinispan.factories.GlobalComponentRegistry - ISPN000128: Infinispan version: Infinispan 'Tactical Nuclear Penguin' 5.3.0.Beta1
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:54:04,448 - INFO org.infinispan.jmx.CacheJmxRegistration - ISPN000031: MBeans were successfully registered to the platform MBean server.
[DEFAULT APPENDER - FROM WEB] 2013-06-13 11:54:07,237 - ERROR org.jgroups.blocks.RequestCorrelator - failed unmarshalling buffer into return value
java.lang.ClassCastException: org.infinispan.commons.hash.MurmurHash3 cannot be cast to java.util.List
    at org.infinispan.distribution.ch.ReplicatedConsistentHash$Externalizer.readObject(ReplicatedConsistentHash.java:145)
    at org.infinispan.distribution.ch.ReplicatedConsistentHash$Externalizer.readObject(ReplicatedConsistentHash.java:135)
    at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:406)
    at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:226)
    at org.infinispan.marshall.jboss.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:164)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
    at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
    at org.infinispan.topology.CacheTopology$Externalizer.readObject(CacheTopology.java:145)
    at org.infinispan.topology.CacheTopology$Externalizer.readObject(CacheTopology.java:134)
    at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:406)
    at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:226)
    at org.infinispan.marshall.jboss.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:164)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
    at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
    at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.readObject(SuccessfulResponse.java:101)
    at org.infinispan.remoting.responses.SuccessfulResponse$Externalizer.readObject(SuccessfulResponse.java:86)
    at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:406)
    at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:226)
    at org.infinispan.marshall.jboss.JBossMarshaller$ExternalizerTableProxy.readObject(JBossMarshaller.java:164)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)
    at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
    at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
    at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:163)
    at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:116)
    at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:104)
    at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)
    at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:390)
    at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:247)
    at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:665)
    at org.jgroups.JChannel.up(JChannel.java:719)
    at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1008)
    at org.jgroups.protocols.RSVP.up(RSVP.java:187)
    at org.jgroups.protocols.FRAG2.up(FRAG2.java:178)
    at org.jgroups.protocols.FlowControl.up(FlowControl.java:404)
    at org.jgroups.protocols.FlowControl.up(FlowControl.java:386)
    at org.jgroups.protocols.tom.TOA.up(TOA.java:122)
    at org.jgroups.protocols.SEQUENCER.up(SEQUENCER.java:251)
    at org.jgroups.protocols.pbcast.GMS.up(GMS.java:893)
    at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
    at org.jgroups.protocols.UNICAST2.handleBatchReceived(UNICAST2.java:838)
    at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:481)
    at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:675)
    at org.jgroups.stack.Protocol.up(Protocol.java:406)
    at org.jgroups.protocols.FD.up(FD.java:274)
    at org.jgroups.stack.Protocol.up(Protocol.java:406)
    at org.jgroups.protocols.MERGE2.up(MERGE2.java:223)
    at org.jgroups.stack.Protocol.up(Protocol.java:406)
    at org.jgroups.protocols.TP.passBatchUp(TP.java:1383)
    at org.jgroups.protocols.TP$BatchHandler.run(TP.java:1519)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

I've burned more than tree days trying to solve that but got nothing, I'll appreciate your help

Edit: when removing the <clustering mode="replication"> everything is working OK.

This sounds like a classpath issue. You should double check all of your jar versions in your lib directories and make sure they all work well together. If your Tomcat app is communicating with JBoss, make sure they all have the same jar versions. With so many lib directories something like this can be quite a pain to debug. :( — Shadow Man, Jun 18 '13 at 00:54
Thanks, I've returned to the last stable version of spring to infinispan cache provider and it looks better, still no clustering but no more wired exceptions — DuduO, Jun 19 '13 at 06:09
Weird. Please note that if you caches for Hibernate 2LC and your own distributed cache for your own code, these need to have different configuration files. Hibernate 2LC has some strict requirements on the cache settings and the named caches that it needs have defined (see examples in https://github.com/galderz/secondlc). Secondly, I'd suggest you try with the latest Infinispan 5.3.0.CR2 release... — Galder Zamarreño, Jun 19 '13 at 08:23

score 0 · Answer 1 · answered Feb 16 '15 at 12:32

The stack trace suggests that one of your nodes is running Infinispan 5.3 (or higher) and the node logging the error is running Infinispan 5.2.x (or lower).

Infinispan doesn't support running nodes with different versions in the same cluster (even if only the micro version is different).

Distributed cache using infinispan on tomcat fails to start

1 Answers1