0

I am new in Mesos and Marathon services. I have setup 3 master and 3 slave server as per www.digitalocean.com. Configured as it is in master servers as well as slaves. Finally I done setup of Mesos, Marathon, Zookeeper and Chronos. Mesos is able to listing with 5050, Marathon is 8080 and Chronos 4400. After few hours my Marthon instances are showing like Error 503 

HTTP ERROR: 503
Problem accessing /. Reason:
    Could not determine the current leader
Powered by Jetty:// 9.3.z-SNAPSHOT.

But mesos is working fine. Every time i am facing this problem and if i restart the marathon service and zookeeper service its working fine.

  • Marathon

    Jun 15 06:19:20 master3 marathon[1054]: INFO Waiting for consistent leadership state. Are we leader?: false, leader: Some(192.168.4.78:8080 (mesosphere.marathon.api.LeaderProxyFilter$:qtp522188921-35)
Jun 15 06:19:20 master3 marathon[1054]: INFO Waiting for consistent leadership state. Are we leader?: false, leader: Some(192.168.4.78:8080 (mesosphere.marathon.api.LeaderProxyFilter$:qtp522188921-35)

  • Zookeeper

    2016-06-15 03:41:13,797 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /192.168.4.78:38339 
2016-06-15 03:41:13,798 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
janisz
  • 6,292
  • 4
  • 37
  • 70
Anilkumar
  • 11
  • 4
  • Could you give more information? What version of Marathon and ZK are you using. Probably useful information could be found in Marathon logs. I suspect when you use latest Marathon version issues will disappear. – janisz Jun 14 '16 at 09:26
  • Especially the following configurations and logs would be helpful: Mesos Master and Zookeeper. – js84 Jun 14 '16 at 11:11
  • getting below log in mesos...at time of down.. E0615 06:18:02.335825 1699 process.cpp:1958] Failed to shutdown socket with fd 37: Transport endpoint is not connected – Anilkumar Jun 15 '16 at 13:21
  • In syslogs.. Jun 15 06:19:20 master3 marathon[1054]: [2016-06-15 06:19:20,575] INFO Waiting for consistent leadership state. Are we leader?: false, leader: Some(192.168.4.78:8080) (mesosphere.marathon.api.LeaderProxyFilter$:qtp522188921-35) Jun 15 06:19:20 master3 marathon[1054]: [2016-06-15 06:19:20,829] INFO Waiting for consistent leadership state. Are we leader?: false, leader: Some(192.168.4.78:8080) (mesosphere.marathon.api.LeaderProxyFilter$:qtp522188921-35) – Anilkumar Jun 15 '16 at 13:25
  • In zookeper log... 2016-06-15 03:41:13,797 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /192.168.4.78:38339 2016-06-15 03:41:13,798 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running – Anilkumar Jun 15 '16 at 13:28
  • But zookeeper is running fine... root@master3:/var/log/zookeeper# service zookeeper status zookeeper start/running, process 1193 – Anilkumar Jun 15 '16 at 13:48
  • Marathon Version 1.1.1 Mesos Version:0.28.1 Zookeeper version: 3.4.5--1, – Anilkumar Jun 16 '16 at 06:09
  • It looks similar to [ZOOKEEPER-2104](https://issues.apache.org/jira/browse/ZOOKEEPER-2104). Could you get some statistics with [`echo ruok | nc 127.0.0.1 5111`](https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html) – janisz Jun 16 '16 at 11:18

0 Answers0