1

We have 2 data centers- --tpmg.cloudapp.net(23.97.52.45) --gns3.cloudapp.net(40.83.12.34)

tmpg has 2 kafka servers kafka1 (172.16.31.142:51134) kafka (172.16.31.143:51135)

kafka server with port 9093 is binded with public azure port 51137 and 2181 with 51138.

I am not able to connect to kafka server in tpmg azure server with my application server in gns3 server.

Can somebody help in this setup!!where is my configuration going wrong!!

logs added when trying to access kafka from gns
2016-05-09 08:48:32,262 INFO or.ap.zo.ClientCnxn [Thread-12-SendThread(23.97.52.45:51138)] Opening socket connection to server 23.97.52.45/23.97.52.45:51138. Will not attempt to authenticate using SASL (unknown error)
2016-05-09 08:48:38,357 INFO or.ap.zo.ZooKeeper [Thread-12] Session: 0x0 closed
2016-05-09 08:48:38,357 INFO or.ap.zo.ClientCnxn [Thread-12-EventThread] EventThread shut down
2016-05-09 08:48:38,358 INFO or.ap.ca.co.ka.KafkaConsumer [Thread-12] Stopping Kafka consumer
2016-05-09 08:48:38,358 ERROR co.mi.mw.op.lo.AppContextLoader [Thread-12] Exception occurred while starting Kafka Subscriber ::
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880) ~[zkclient-0.3.jar:0.3]
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98) ~[zkclient-0.3.jar:0.3]
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84) ~[zkclient-0.3.jar:0.3]
    at kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:171) ~[kafka_2.9.2-0.8.2.1.jar:?]
    at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:126) ~[kafka_2.9.2-0.8.2.1.jar:?]
    at kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:66) ~[kafka_2.9.2-0.8.2.1.jar:?]
    at kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:69) ~[kafka_2.9.2-0.8.2.1.jar:?]
    at kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:105) ~[kafka_2.9.2-0.8.2.1.jar:?]
    at kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala) ~[kafka_2.9.2-0.8.2.1.jar:?]
    at org.apache.camel.component.kafka.KafkaConsumer.doStart(KafkaConsumer.java:61) ~[camel-kafka-custom-1.0.jar:?]
    at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.startService(DefaultCamelContext.java:2083) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRouteConsumers(DefaultCamelContext.java:2377) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.doStartRouteConsumers(DefaultCamelContext.java:2313) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:2243) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:2256) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.startRouteService(DefaultCamelContext.java:2133) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.startRoute(DefaultCamelContext.java:794) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:2109) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.addRouteDefinitions(DefaultCamelContext.java:726) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.builder.RouteBuilder.populateRoutes(RouteBuilder.java:337) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:264) ~[camel-core-2.13.4.jar:2.13.4]
    at org.apache.camel.impl.DefaultCamelContext.addRoutes(DefaultCamelContext.java:688) ~[camel-core-2.13.4.jar:2.13.4]

Thanks in advance.

Nilotpal
  • 3,237
  • 4
  • 34
  • 56
  • It will be be helpful if you post the exception/error you were getting? – avr May 09 '16 at 18:27
  • Are you using Kafka on Windows or Linux VMs ? Because Windows VMs do have Firewall preconfigured which is blocking your Kafka server. – astaykov May 09 '16 at 20:05
  • 1
    @astaykov i am using linux VM's – Nilotpal May 10 '16 at 05:10
  • 1
    @avr : added logs. – Nilotpal May 10 '16 at 05:23
  • According to this log message `org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000` your `zookeeper` is not accessible from `gsn` machine. So make sure your kafka port and zookeeper port are opened for `gsn` machine! – avr May 10 '16 at 07:40
  • Thanks avr :) Yes zookeeper is accessible from gns machine. can you tell like how to increase the zookeeper timeout from java code. – Nilotpal May 10 '16 at 08:02

0 Answers0