GemFire with GFSH - Region creation with server start throws BindException

Question

I am trying to create a GemFire cache and region at the time of starting a cache server using GFSH command.

GFSH:

gfsh start server --name=server1 --server-port=40405 --classpath=$CLASSPATH   --cache-xml=/tmp/gemfire/8.2.7/config/cache.xml  --locators=hostA[10334],hostB[10334] --mcast-port=0

Cache.xml:

<?xml version="1.0" encoding="UTF-8"?><cache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schema.pivotal.io/gemfire/cache" xsi:schemaLocation="http://schema.pivotal.io/gemfire/cache http://schema.pivotal.io/gemfire/cache/cache-8.1.xsd" version="8.1" lock-lease="120" lock-timeout="60" search-timeout="300" is-server="false" copy-on-read="false">
 <pdx>
    <pdx-serializer>
      <class-name>
       com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer
      </class-name>
    <parameter name="classes">
      <string>com.gemfire.DomainObjects</string>
    </parameter>
  </pdx-serializer>
 </pdx>
<region name="Customer" refid="REPLICATE"></region>
</cache>

When I run the gfsh command, I am getting below exception.

Caused by: java.net.BindException: Failed to create server socket on  null[40,405]
        at com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:828)
        at com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:758)
        at com.gemstone.gemfire.internal.cache.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:466)
        at com.gemstone.gemfire.internal.cache.BridgeServerImpl.start(BridgeServerImpl.java:342)
        at com.gemstone.gemfire.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:607)
        ... 11 more
Caused by: java.net.BindException: Address already in use
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
        at java.net.ServerSocket.bind(ServerSocket.java:375)
        at com.gemstone.gemfire.internal.SocketCreator.createServerSocket(SocketCreator.java:825)
        ... 15 more

Any help?

I used netstat to find if the port is occupied. I couldnt find any and i executed the script still i am getting same exception.

bash-4.1$ netstat | grep 40405
bash-4.1$ ./startServer.sh 
.......................................................................................................................................................................................................................................................The Cache Server process terminated unexpectedly with exit status 1. Please refer to the log file in /var/tmp/sn17180/gemfire/8.2.7/config/server1 for full details.

[severe 2018/05/06 04:37:53.156 IST  libgemfire.so nid=0x17921700] SIGQUIT received, dumping threads  
java.io.EOFException: Locator at hostB(server1:18877)<v73>:10334 did not respond. This is normal if the locator was shutdown. If it wasn't check its log for exceptions.
        at com.gemstone.org.jgroups.stack.tcpserver.TcpClient.requestToServer(TcpClient.java:125)
        at com.gemstone.org.jgroups.stack.tcpserver.TcpClient.requestToServer(TcpClient.java:78)
        at com.gemstone.gemfire.internal.cache.ClusterConfigurationLoader.requestConfigurationFromLocators(ClusterConfigurationLoader.java:171)
        at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.requestAndApplySharedConfiguration(GemFireCacheImpl.java:874)
        at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.init(GemFireCacheImpl.java:1025)
        at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:688)
        at com.gemstone.gemfire.cache.CacheFactory.create(CacheFactory.java:182)
        at com.gemstone.gemfire.cache.CacheFactory.create(CacheFactory.java:229)
        at com.gemstone.gemfire.distributed.ServerLauncher.startWithGemFireApi(ServerLauncher.java:793)
        at com.gemstone.gemfire.distributed.ServerLauncher.start(ServerLauncher.java:695)
        at com.gemstone.gemfire.distributed.ServerLauncher.run(ServerLauncher.java:625)
        at com.gemstone.gemfire.distributed.ServerLauncher.main(ServerLauncher.java:195)
Exception in thread "main" com.gemstone.gemfire.GemFireIOException: While starting bridge server  CacheServer on port=40405 client subscription config policy=none client subscription config capacity=1 client subscription config overflow directory=.
        at com.gemstone.gemfire.internal.cache.xmlcache.CacheCreation.create(CacheCreation.java:611)
        at com.gemstone.gemfire.internal.cache.xmlcache.CacheXmlParser.create(CacheXmlParser.java:340)
        at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.loadCacheXml(GemFireCacheImpl.java:4269)
        at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.initializeDeclarativeCache(GemFireCacheImpl.java:1184)
        at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.init(GemFireCacheImpl.java:1026)
        at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:688)
        at com.gemstone.gemfire.cache.CacheFactory.create(CacheFactory.java:182)
        at com.gemstone.gemfire.cache.CacheFactory.create(CacheFactory.java:229)
        at com.gemstone.gemfire.distributed.ServerLauncher.startWithGemFireApi(ServerLauncher.java:793)
        at com.gemstone.gemfire.distributed.ServerLauncher.start(ServerLauncher.java:695)
        at com.gemstone.gemfire.distributed.ServerLauncher.run(ServerLauncher.java:625)
        at com.gemstone.gemfire.distributed.ServerLauncher.main(ServerLauncher.java:195)
Caused by: java.net.BindException: Failed to create server socket on  null[40,405]

Possible duplicate of [BindException: Address already in use even with unique port](https://stackoverflow.com/questions/23329580/bindexception-address-already-in-use-even-with-unique-port) — Robin Green, May 04 '18 at 21:28
No. this is different. This issue is common across any frameworks we use. But as far as gemfire is concerned, I am a beginner and i am not sure if i am doing things right.Thats why i have posted this question. — Sudharsan, May 05 '18 at 23:17

John Blum · Accepted Answer · 2018-05-08T17:12:15.870

0

This means some other process running on localhost is already bound and listening to client socket connections on port 40405. Perhaps you have another GemFire Server already running on that port?

In which case, you need to vary the port numbers for as many servers as you start, for example...

gfsh> start server --name=One --server-port=40405 ...
...
gfsh> start server --name=Two --server-port=40406 ...
...
gfsh> start server --name=Three --server-port=40407 ...
...

... and so and so forth.

If a non-GemFire process is already running using that port number than either considering killing that process or using another port for your GemFire CacheServers.

You can use the Linux netstat command to find which process is using the port and act appropriately.

edited May 08 '18 at 17:12

answered May 04 '18 at 21:51

John Blum

7,381
1
20
30

Thanks for your reply. Is this the right way of creating cache and region while starting up the server?. Can you please suggest a tutorial for using cache.xml (to create cache and regions) with gfsh server start and then have another Java process to load the regions created. I am finding it hard to find such resource. – Sudharsan May 05 '18 at 22:56
I have updated the post with netstat. I find the port is free. But i have got no clue why this issue is coming. – Sudharsan May 05 '18 at 23:19
I have got it fixed. I killed PIDs and tried but it didnt work. Then i removed all server and locator directories, then re-executed the steps which worked. Do these directories create issue?. Should we clear the directories before server restart? – Sudharsan May 06 '18 at 00:37
It's possible that you had some duplication causing two instances of the server to be started. – Robin Green May 06 '18 at 05:46
If your clients are connecting through locator(s), you can also specify an ephemeral port for the server (`--server-port=0`). That way the server will pick some random open port. Clients will discover the correct port when they initially connect to the locator. Note the option is `--server-port` and not `--port` as indicated above. `--port` is used to set the port when starting a locator. – Jens D May 08 '18 at 13:20
Right, my bad. Thanks Jens. It is `--server-port` option in the `start-server` _Gfsh_ command to affect the `CacheServer` port. The `--port` affects the p2p port used to communicate amongst (peers, or `CacheServers`) forming a cluster. – John Blum May 08 '18 at 17:13

GemFire with GFSH - Region creation with server start throws BindException

1 Answers1