Cluster with Infinispan Server 7.0 and Embedded in JBoss AS7

Question

I'm having issues clustering Infinispan (via JGroups) across different servers. When I try this kind of cluster:

cache-container configured in subsystem config on Infinispan 7 standalone server 1 (based on Wildfly)
cache-container configured in subsystem config on Infinispan 7 standalone server 2 (based on Wildfly)

...each node joins the cluster fine (I can see the via the output produced by I'm assuming JGroups).

However, when I try this kind of cluster:

cache-container configured in infinispan.xml deployed along with Infinispan 7 dep jars on JBoss AS 7 server 1
cache-container configured in subsystem config on Infinispan 7 standalone server 2 (based on Wildfly)

...the node deployed with the app into JBoss AS 7 doesn't join the cluster.

In short, it seems like a node deployed with an application (i.e. embedded Infinispan jars and config in a war) can't join a cluster with a node deployed standalone (cache-container configured in xml via subsystem config).

To note, I'm using Infinispan 7 (I acknowledge its in beta right now, but I need to use it because the JCache dependencies in Infinispan 6 unfortunately point to the 1.0.0 PFD version of the cache-api dependency (JSR 107).

I'll also note that the reason I'm deploying Infinispan along with my application is that the server I'm deploying to won't necessarily have Infinispan available (i.e. JBoss AS 7 or very likely WebSphere in some cases). Someone could tell me that its more appropriate to configure Infinispan on the app server and not deploy it with my app, but after trying for 1+ hours to configure JBoss AS 7 with an Infinispan 7 subsystem, I gave up. My above deployment scenario seems legit....unless there is another suggestion?

JGroups config I'm deploying with my application:

<config xmlns="urn:org:jgroups" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.4.xsd">

    <TCP bind_port="7802" />
    <TCPPING timeout="3000" initial_hosts="${jgroups.tcpping.initial_hosts:MY_INTERNAL_DOMAIN[7800],localhost[7801],localhost[7802]}" port_range="1"
    num_initial_members="3" />
    <VERIFY_SUSPECT timeout="1500" />
    <pbcast.NAKACK use_mcast_xmit="false" retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true" />
    <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="400000" />
    <pbcast.GMS print_local_addr="true" join_timeout="3000" view_bundling="true" />

</config>

JGroups stack I've configured in Infinispan 7 standalone server:

<stack name="tcp">          
    <transport type="TCP">
        <property name="bind_port">7800</property>
    </transport>
    <protocol type="TCPPING">
        <property name="timeout">3000</property>
        <property name="initial_hosts">${jgroups.tcpping.initial_hosts:MY_INTERNAL_DOMAIN.atldev.com[7800],localhost[7801]}</property>
        <property name="port_range">1</property>
        <property name="num_initial_members">3</property>
    </protocol>
    <protocol type="VERIFY_SUSPECT">
        <property name="timeout">1500</property>
    </protocol>
    <protocol type="pbcast.NAKACK">
        <property name="use_mcast_xmit">false</property>
        <property name="retransmit_timeout">300,600,1200,2400,4800</property>
        <property name="discard_delivered_msgs">true</property>
    </protocol>
    <protocol type="pbcast.STABLE">
        <property name="stability_delay">1000</property>
        <property name="desired_avg_gossip">50000</property>
        <property name="max_bytes">400000</property>
    </protocol>
    <protocol type="pbcast.GMS">
        <property name="print_local_addr">true</property>
        <property name="join_timeout">3000</property>
        <property name="view_bundling">true</property>
    </protocol>
</stack>

Cache-container configured in Infinispan 7 standalone server:

    <subsystem xmlns="urn:infinispan:server:core:7.0" default-cache-container="ervm-caches">
        <cache-container name="ervm-caches" default-cache="ervm-default-cache">
            <transport stack="tcp" cluster="ervm-cluster"/>
            <distributed-cache name="default" mode="SYNC" segments="20" owners="2" remote-timeout="30000" start="EAGER">
                <locking acquire-timeout="30000" concurrency-level="1000" striping="false"/>
                <transaction mode="NONE"/>
            </distributed-cache>
            <distributed-cache name="memcachedCache" mode="SYNC" segments="20" owners="2" remote-timeout="30000" start="EAGER">
                <locking acquire-timeout="30000" concurrency-level="1000" striping="false"/>
                <transaction mode="NONE"/>
            </distributed-cache>
            <replicated-cache name="ervm-default-cache" mode="SYNC" start="EAGER"/>
        </cache-container>
    </subsystem>

Cache-container configured and deployed in my application (war) to JBoss AS 7:

<?xml version="1.0" encoding="UTF-8"?>
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:infinispan:config:7.0 http://www.infinispan.org/schemas/infinispan-config-7.0.xsd"
    xmlns="urn:infinispan:config:7.0">

    <jgroups>
        <stack-file name="tcp" path="ervm-jgroups-tcp.xml" />
        <stack-file name="udp" path="ervm-jgroups-udp.xml" />
    </jgroups>

    <cache-container name="ervm-caches" default-cache="ervm-default-cache">
        <transport stack="tcp" cluster="ervm-cluster" />
        <replicated-cache name="ervm-default-cache" mode="SYNC" start="EAGER" />
    </cache-container>

</infinispan>

I just tried to deploy to different Wildfly servers (on different IPs) with the subsystem configuration similar to what was previously mentioned and can't get them to join a cluster together either!! — sparty02, Sep 16 '14 at 21:06
I just found http://stackoverflow.com/questions/24068586/infinispan-how-to-combine-embedded-cache-and-standalone-server-in-one-cluster?rq=1 , which sounds a lot like what I'm trying to do. I'm going to give that a whirl. — sparty02, Sep 17 '14 at 18:39

score 0 · Answer 1 · answered Sep 17 '14 at 14:48

I am not sure whether this helps, but you should be rather using socket binding for specifying the ports etc.:

 <subsystem xmlns="urn:jboss:domain:jgroups:1.2" default-stack="udp">
     <stack name="udp">
        <transport type="UDP" socket-binding="jgroups-udp">
        ...
     </stack>
 </subsystem>
 <interfaces>
     <interface name="public">
        <inet-address value="${iface.public}" />
     </interface>
     <interface name="clustering">
        <inet-address value="${iface.clustering}" />
     </interface>
 </interfaces>
 <socket-binding-group name="standard-sockets" default-interface="public">
     <socket-binding name="jgroups-udp" interface="clustering" multicast-address="${udpGroup}" multicast-port="45688" port="55200" />
     <socket-binding name="jgroups-udp-fd" interface="clustering" port="54200" />

Also, if you want to combine access through protocols (HotRod, Memcached...) and embedded access, you should enable the compatibility mode.

Cluster with Infinispan Server 7.0 and Embedded in JBoss AS7

1 Answers1