Multiple gemfire cache-server on same host machine

Question

Hi i want to start more than one gemfire cache-server on same host using Spring gemfire 8.1.Please find below gemfire configuration file. I want to start GFServer1 and GFServer2 on same host i.e.HOSTNAME using Spring Gemfire configuration. I want to avoid gfsh command and start everything from eclipse and connect client to these servers on the same host. Thanks in advance

    <util:properties id="gemfireProperties">
        <prop key="name">Locator_Dev</prop>
        <prop key="mcast-port">0</prop>
        <prop key="locators">HOSTNAME[1099]</prop>
        <prop key="log-level">warning</prop>
        <prop key="http-service-port">8181</prop>
        <prop key="jmx-manager">true</prop>
        <prop key="jmx-manager-port">1199</prop>
        <prop key="jmx-manager-start">true</prop>
        <prop key="start-locator">HOSTNAME[1099]</prop>
    </util:properties>

    <gfe:cache properties-ref="gemfireProperties" />


    <gfe:cache-server id="GFServer1" auto-startup="true"
        bind-address="HOSTNAME" port="40411" host-name-for-clients="HOSTNAME"
        load-poll-interval="2000" max-connections="22" max-threads="16"
        max-message-count="1000" max-time-between-pings="30000" >

        <gfe:subscription-config eviction-type="ENTRY"
            capacity="1000" disk-store="diskStore1" />
    </gfe:cache-server>


    <gfe:cache-server id="GFServer2" auto-startup="true"
        bind-address="HOSTNAME" port="40412" host-name-for-clients="HOSTNAME"
        load-poll-interval="2000" max-connections="22" max-threads="16"
        max-message-count="1000" max-time-between-pings="30000" >

        <gfe:subscription-config eviction-type="ENTRY"
            capacity="1000" disk-store="diskStore1" />
    </gfe:cache-server>


   <gfe:disk-store id="diskStore1" queue-size="50"
        auto-compact="true" max-oplog-size="10" time-interval="9999">
        <gfe:disk-dir
            location="D:\NP\WorkSpace\GemfireRegionSolutionNStart\disk-store\store_1"
            max-size="20" />
        <gfe:disk-dir
            location="D:\NP\WorkSpace\GemfireRegionSolutionNStart\disk-store\store_2"
            max-size="20" />
    </gfe:disk-store>


    <gfe:replicated-region id="customer" name="Customer">
    </gfe:replicated-region>

    <gfe:replicated-region id="bookMaster" name="BookMaster">
    </gfe:replicated-region>

</beans> 

Answer 1

The configuration you have posted will create two cache-servers within the same JVM, i.e it will open up two ports within the same process.

If this is not what you want, i.e you want two distinct process, in eclipse you will have to provide two runtime configurations to start the two servers.

Answer 2

Is there a specific question? As @Swapnil points out, this will start 2 GemFire "Cache Servers" (ServerSockets listening for Cache Clients) as you have appropriately configured on the same host within the same JVM. This will work regardless of how this is executed (i.e. IDE, command-line, from Gfsh or from Spring Boot).

Let us know if you have a more specific question, thanks!

Answer 3

So you can configure the LocatorLauncherFactoryBean, for example, like so...

<uti:properties id="gemfireProperties">
  <prop key="log-level">config</prop>
  <prop key="http-service-port">8181</prop>
  <prop key="jmx-manager">true</prop>
  <prop key="jmx-manager-port">1199</prop>
  <prop key="jmx-manager-start">true</prop>
  <prop key="locators">host1[10334],host2[11235],...,hostN[20668]</prop>
</util:properties>

<bean id="locator" class="org.spring.data.gemfire.config.LocatorLauncherFactoryBean">
  <property name="gemfireProperties" ref="gemfireProperties"/>
  <property name="memberName" value="SpringDataGemFireLocator"/>
  <property name="bindAddress" value="10.124.12.24"/>
  <property name="port" value="12480"/>
</bean>

As you may have noticed, this Locator can join other Locators in the GemFire Cluster, which were specified in the "gemfireProperties" bean with the "locators" GemFire System property.

NOTE: the "bindAddress" property to the LocatorLauncherFactoryBean is only necessary if the localhost where this Locator will be running has multiple NICs and you want to bind to a specific NIC.

Also, I have set the JMX Manager GemFire System properties to enable the Locator to become and actually start a Manager (on port 1199). This allows you to connect to this Locator from Gfsh either with gfsh>connect --locator=localhost[12480] or with gfsh>connect --jmx-manager=localhost[1199].

Basically, the "gemfireProperties" bean allows you to configure any valid GemFire System property.

Now, since this Locator is running from within your IDE, you will need to configure the "run profile" with a $GEMFIRE environment variable pointing at a GemFire distribution downloaded from Pivotal's website in order to get Pulse running from this Locator. This is expected by the GemFire Manager's ManagementAgent when making a decision of whether to 1. start the embedded HTTP Service (Jetty) running GemFire's out-of-box webapps (e.g. Pulse) and 2. whether it can find Pulse and start the webapp. The ManagementAgent looks for Pulse in the distro.

For instance, I set my $GEMFIRE environment variable to...

/Users/jblum/Downloads/Pivotal/GemStone/Products/GemFire/Pivotal_GemFire_820_b17919_Linux

Now, to get your individiual Spring-configured GemFire Servers to connect to the cluster, that is simple.

Again, you only need a "gemfireProperties" bean defined in each Spring GemFire Server XML configuration file with the "locators" GemFire System property defined, e.g. ...

<uti:properties id="gemfireProperties">
  <prop key="log-level">config</prop>
  <prop key="locators">localhost[12480]</prop>
</util:properties>

<gfe:cache properties-ref="gemfireProperties"/>

This configuration will enable the GemFire Data Nodes to connect to the cluster, and this cluster will be visible from Pulse, if everything is setup correctly.

Again, hope this helps.

Cheers, John

Answer 4

OK... so you have a few options from within your IDE (e.g. Eclipse).

If you break the Spring config above into 2 separate Spring (Data GemFire) XML configuration files, each containing 1 of the <gfe:cache-server> elements, call these files, for example, spring-gemfire-server1-context.xml and spring-gemfire-server2-context.xml, you could run the Spring-based/configure GemFire "data nodes" and Cache Servers with the following...

SimpleSpringApplication

Where the argument to the simple little Java "main" program is the file system path to 1 of the Spring XML configuration files noted above.

Better yet, you can launch these Spring (GemFire) configs with a SpringBoot application using something like...

UsefulSpringBootGemFireApplication

As you can see here, you can use either the @Import annotation if you are configuring GemFire from Spring using Java-based Configuration (for example, UsefulSpringBasedGemFireConfiguration), or in your case, with XML using the @ImportResoruce("/classpath/to/spring-gemfire-server1-context.xml") annotation.

It maybe possible (not sure) to pass a System property value into the @ImportResource annotation, like so...

@SpringBootApplication
@ImportResource("${spring-gemfire-context-xml-location}")
class SpringBootGemFireApplication {
  ...
}

Then, from within your IDE, you can create 2 separate "run profiles" using the same SpringBootGemFireApplication class and then setting the System property (e.g. -Dspring-gemfire-context-xml-location=/path/to/spring-gemfire-server1-context.xml) appropriately for each configuration.

Remember, you can use Spring's ResourceLoader path qualifiers (e.g. file:, http:, etc) to resolve your Spring GemFire config from different source locations (CLASSPATH, file system, etc)... see the table ("Table 7.1 Resource strings") in hyperlinked section of the Spring Framework Reference Guide.

In the worst case, you need to create 2 separate, but nearly identical SpringBoot application Java main classes to launch your 2 configuration files for each GemFire Data Node (& CacheServer) JVM process. However, hopefully using the System property approach allows you to recycle the same class in 2 separate run profiles.

So, this leaves you with 2 GemFire Data Node JVM processes now. But, what about he Locator?

Well, you can continue to embed the Locator in the GemFire Data Node Server JVM process as before, but if you really want a standalone Locator JVM process, then you can use the following, "experimental" class...

LocatorLauncherFactoryBean

This class uses GemFire's LocatorLauncher class to configure and bootstrap a GemFire Locator from Spring config. I created this class over 2 years ago for a customer POC as example of how a developer might configure a GemFire Locator from Spring config.

The Spring XML configuration (used by the SpringBootGemFireApplication @ImportResoruce annotation, in yet another "run profile" and System property combination), would look similar to the following...

locator.xml

Now, you have effectively achieved 3 separate GemFire JVM processes (2 Servers and 1 Locator). You can scale this from with our IDE to however many Servers and Locators you want, and Pulse will show all of these in the cluster, providing the cluster is configured correctly (Servers pointed at Locators, etc).

Hope this helps.

Cheers! John