Hazelcast Node multiple errors when run multiple nodes

Question

i'm trying to launch several nodes with this programmatic configuration

    public class Server {

    private Server() {
        //TODO: MapStore/MapLoader config here
        MapConfig accountMapConfig = new MapConfig(PropertiesLoader.ACCOUNT_MAP);
        MapStoreConfig accountStoreConfig = new MapStoreConfig();
        accountStoreConfig.setEnabled(true)
                .setClassName("com.hazelcast.certification.domainstore.AccountStore")
                .setInitialLoadMode(MapStoreConfig.InitialLoadMode.EAGER);
        accountMapConfig.setMapStoreConfig(accountStoreConfig);
        
        MapConfig merchantMapConfig = new MapConfig(PropertiesLoader.MERCHANT_MAP);
        MapStoreConfig merchantStoreConfig = new MapStoreConfig();
        merchantStoreConfig.setEnabled(true)
                .setInitialLoadMode(MapStoreConfig.InitialLoadMode.EAGER)
                .setClassName("com.hazelcast.certification.domainstore.MerchantStore");
        merchantMapConfig.setMapStoreConfig(merchantStoreConfig);
        
        MapConfig mRules = new MapConfig();
        mRules.setName(PropertiesLoader.RULESRESULT_MAP);
        
        //TODO: Config for WAN Replication
        
        
        Config platform_config = new Config();
        platform_config.setClusterName("source");
        NetworkConfig networkConfig = platform_config.getNetworkConfig();
        JoinConfig join = networkConfig.getJoin();
        join.getMulticastConfig().setEnabled(false);
        InterfacesConfig interfaceConfig = networkConfig.getInterfaces();
        interfaceConfig.setEnabled( true )
                    .addInterface( "192.168.0.13" );
        
        
        //TODO: Platform config here
        platform_config.setLicenseKey(License.KEY_HE);
        
        
        
        platform_config.addMapConfig(accountMapConfig);
        platform_config.addMapConfig(merchantMapConfig);
        platform_config.addMapConfig(mRules);
        
       
        
        platform_config.getMapConfig(PropertiesLoader.ACCOUNT_MAP)
        .setWanReplicationRef(new WanReplicationRef().setName("wololo"));
        platform_config.getMapConfig(PropertiesLoader.MERCHANT_MAP)
        .setWanReplicationRef(new WanReplicationRef().setName("wololo"));
        platform_config.getMapConfig(PropertiesLoader.RULESRESULT_MAP)
        .setWanReplicationRef(new WanReplicationRef().setName("wololo"));
        
        WanReplicationConfig wanReplicationConfig = new WanReplicationConfig()
                .setName("wololo");
        WanBatchPublisherConfig batchPublisherConfig = new WanBatchPublisherConfig()
                .setClusterName("jet")
                .setTargetEndpoints("192.168.0.13:5703");
        wanReplicationConfig.addBatchReplicationPublisherConfig(batchPublisherConfig);
        platform_config.addWanReplicationConfig(wanReplicationConfig);
    
        
        

        Hazelcast.newHazelcastInstance(platform_config);
    }

    public static void main(String[] args) {
        
        
        new Server();
        
    }
    
    
    
    
    
        
    
}

and i don't know why i'm getting 2 main errors when i start instances.

First one:

SEVERE: [192.168.0.13]:5701 [source] [4.3] Service with name 'hz:impl:jetService' not found!
com.hazelcast.core.HazelcastException: Service with name 'hz:impl:jetService' not found!
    at com.hazelcast.spi.impl.NodeEngineImpl.getService(NodeEngineImpl.java:377)
    at com.hazelcast.spi.impl.operationservice.Operation.getService(Operation.java:409)
    at com.hazelcast.jet.impl.operation.AsyncOperation.beforeRun(AsyncOperation.java:45)
    at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:214)
    at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:406)
    at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:433)
    at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:596)
    at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:581)
    at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:540)
    at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:237)
    at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:59)
    at com.hazelcast.client.impl.protocol.task.AbstractInvocationMessageTask.processInternal(AbstractInvocationMessageTask.java:38)
    at com.hazelcast.client.impl.protocol.task.AbstractAsyncMessageTask.processMessage(AbstractAsyncMessageTask.java:71)
    at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.initializeAndProcessMessage(AbstractMessageTask.java:152)
    at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.run(AbstractMessageTask.java:115)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
    at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:64)
    at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:80)
Caused by: com.hazelcast.spi.exception.ServiceNotFoundException: Service with name 'hz:impl:jetService' not found!
    ... 20 more

and the Second One:

i dont know why im getting spam and more spams about the connection trying to connect the node with a certain address an endless loop

oct. 19, 2021 10:56:51 A. M. com.hazelcast.client.impl.ClientEngine
INFO: [192.168.0.13]:5701 [source] [4.3] Applying a new client selector :ClientSelector{any}
oct. 19, 2021 10:56:52 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=3, /192.168.0.13:5701->/192.168.0.13:52705, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 6930d85e-fe3b-4758-8712-8015a6022fc4, authentication failed
oct. 19, 2021 10:56:52 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=3, /192.168.0.13:5701->/192.168.0.13:52705, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:52 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=4, /192.168.0.13:5701->/192.168.0.13:52707, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid f6772096-f34b-4292-88db-d809d070604d, authentication failed
oct. 19, 2021 10:56:52 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=4, /192.168.0.13:5701->/192.168.0.13:52707, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:53 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=5, /192.168.0.13:5701->/192.168.0.13:52711, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 6930d85e-fe3b-4758-8712-8015a6022fc4, authentication failed
oct. 19, 2021 10:56:53 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=5, /192.168.0.13:5701->/192.168.0.13:52711, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:54 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=6, /192.168.0.13:5701->/192.168.0.13:52712, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 6930d85e-fe3b-4758-8712-8015a6022fc4, authentication failed
oct. 19, 2021 10:56:54 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=6, /192.168.0.13:5701->/192.168.0.13:52712, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:55 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=7, /192.168.0.13:5701->/192.168.0.13:52715, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 6930d85e-fe3b-4758-8712-8015a6022fc4, authentication failed
oct. 19, 2021 10:56:55 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=7, /192.168.0.13:5701->/192.168.0.13:52715, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:56 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=8, /192.168.0.13:5701->/192.168.0.13:52718, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 6930d85e-fe3b-4758-8712-8015a6022fc4, authentication failed
oct. 19, 2021 10:56:56 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=8, /192.168.0.13:5701->/192.168.0.13:52718, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:56 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=9, /192.168.0.13:5701->/192.168.0.13:52719, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 6930d85e-fe3b-4758-8712-8015a6022fc4, authentication failed
oct. 19, 2021 10:56:56 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=9, /192.168.0.13:5701->/192.168.0.13:52719, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:57 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=10, /192.168.0.13:5701->/192.168.0.13:52721, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 6930d85e-fe3b-4758-8712-8015a6022fc4, authentication failed
oct. 19, 2021 10:56:57 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=10, /192.168.0.13:5701->/192.168.0.13:52721, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:58 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=11, /192.168.0.13:5701->/192.168.0.13:52722, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 6930d85e-fe3b-4758-8712-8015a6022fc4, authentication failed
oct. 19, 2021 10:56:58 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=11, /192.168.0.13:5701->/192.168.0.13:52722, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:58 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=12, /127.0.0.1:5701->/127.0.0.1:52724, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid 244377e7-395e-4972-9a3c-b9d03fcdec50, authentication failed
oct. 19, 2021 10:56:58 A. M. com.hazelcast.internal.nio.tcp.TcpIpConnection
INFO: [192.168.0.13]:5701 [source] [4.3] Connection[id=12, /127.0.0.1:5701->/127.0.0.1:52724, qualifier=null, endpoint=null, alive=false, connectionType=NONE] closed. Reason: Connection closed by the other side
oct. 19, 2021 10:56:58 A. M. com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
WARNING: [192.168.0.13]:5701 [source] [4.3] Received auth from Connection[id=13, /127.0.0.1:5701->/127.0.0.1:52726, qualifier=null, endpoint=null, alive=true, connectionType=NONE] with clientUuid c1c696af-7e7c-4e63-8062-240baeca9c7f, authentication failed

I DONT WANT A SOLUTION, i just want to know is this happening when im trying to start a node.

I guess you're mixing Jet and IMDG clusters. You should use IMDG nodes and all clients, or use Jet nodes and clients. For wan replication, the remote cluster also has to be of the same type. — Oliv, Oct 19 '21 at 10:16
Im sorry @Oliv im a bit confused, maybe because i'm new ¿Where am i using hazelcast jet? because it's everything on IMDG, even wan replication it's for IMDG, thank you for your reply ¿Can you tell me where am i using hazelcast jet tools? because it's supossed to be 100% imdg this class. — killopolo, Oct 19 '21 at 10:37
`[4.3]` in your logs must be a Jet release number, as the highest IMDG 4.* release is currently `4.2.2`. So somewhere on your classpath you have Jet. If you can, try 5.0. Also watch out for `config.getNetworkConfig().getJoin().getAutoDetectionConfig()`. But if you're using WAN then you are a commercial customer so should raise a support ticket. — Neil Stevenson, Oct 19 '21 at 13:48
@NeilStevenson do you mean the logs.... well you mean the dependency... my pom says i have this ` com.hazelcast.jet hazelcast-jet-enterprise 4.3 ` and Hazelcast has this ` com.hazelcast hazelcast-enterprise-all 4.1.1 ` does this has anything to do with my log problem? — killopolo, Oct 20 '21 at 08:23
Jet 4.3 embeds Hazelcast 4.03 and you also explictly have Hazelcast 4.1.1. So which one is used will depend on the ordering. Remove one and that doubt disappears. Both of these aren't the latest, so if you're making changes anyway some form of upgrade wouldn't be a bad idea. — Neil Stevenson, Oct 22 '21 at 07:25

score 0 · Accepted Answer · answered Oct 22 '21 at 07:31

Your source cluster is defined as

        Config platform_config = new Config();
        platform_config.setClusterName("source");
        interfaceConfig.setEnabled( true )
                    .addInterface( "192.168.0.13" );

so presumably runs on 192.168.0.13.

Your target cluster has this definition

        WanBatchPublisherConfig batchPublisherConfig = new WanBatchPublisherConfig()
                .setClusterName("jet")
                .setTargetEndpoints("192.168.0.13:5703");

So both are running on the same host.

This is fine, but the source cluster has no target port range specified, so will default to probing 5701 upwards when it starts. So it will find the target cluster on 5703, and try to join it as the same cluster rather than as two WAN connected clusters.

The easier solution is to set a port range for each cluster that doesn't overlap. Use config.getNetworkConfig().setPort(i)

That's one of the answers that literally i could found out later... but the principal answer for this question is the mancenter, the mancenter throws every error i had, i had to close the man center because everytime i got connection problems for some reason in every version of management center, i already found out too the error "Service with name 'hz:impl:jetService' not found!" was because hazelcast use the jet instance for some reason, you just have to delete the jet dependency or look in the jar library and delete that dependency. — killopolo, Oct 29 '21 at 08:51

Hazelcast Node multiple errors when run multiple nodes

1 Answers1