2

I am running — currently on a single machine — one Mesos master, one slave, one Zookeeper and Marathon. All are running inside a Docker container. They all seem to be communicating with each other correctly.

NOTE: Usually Mesos master runs on port 5050 and slaves on port 5051. The port 5050 was already used by some other app on my machine, so master runs on port 5051 and slave on port 5052.

Through Marathon, I then try to run the basic-0 example, but the job keeps failing. I used to be able to run jobs, but I made some changes to the way I was running the Docker containers, in order to remove all local ips.

I run Mesos containers with the following parameters :

Master :

docker run --net=host -d \
       -e MESOS_NATIVE_JAVA_LIBRARY="/usr/local/lib/libmesos.so" \
       -e MESOS_NATIVE_LIBRARY="/usr/local/lib/libmesos.so" \
       -e MESOS_QUORUM=1 -e MESOS_LOG_DIR='/var/tmp' \
       -e MESOS_WORK_DIR='/tmp' \
       -e MESOS_ZK=zk://$IP:2181/mesos \
       -e MESOS_PORT=5051 \
       -e MESOS_ADVERTISE_PORT=5051 \
       -e MESOS_ADVERTISE_IP=$IP \
       -e MESOS_HOSTNAME=$IP \
       -p 5051:5051 \
       eurobd/mesos-master

Slave :

docker run --net=host -d \
       -e MESOS_MASTER=zk://$IP:2181/mesos \
       -e MESOS_LOG_DIR=/var/tmp \
       -e MESOS_EXECUTOR_REGISTRATION_TIMEOUT=2mins \
       -e MESOS_ISOLATOR=cgroups/cpu,cgroups/mem \
       -e MESOS_CONTAINERIZERS=docker,mesos \
       -e MESOS_HOSTNAME=$IP \
       -e MESOS_ADVERTISE_IP=$IP \
       -e MESOS_IP=$IP \
       -e MESOS_ADVERTISE_PORT=5052 \
       -e MESOS_PORT=5052 \
       -v /run/docker.sock:/run/docker.sock \
       -v /sys:/sys \
       -v /proc:/host/proc:ro \
       -p 5052:5052 \
       eurobd/mesos-slave

Here are the various logs that I read, but didn't help me :

Executor stderr :

I0517 10:00:48.011389   125 logging.cpp:188] INFO level logging started!
I0517 10:00:48.012583   125 exec.cpp:143] Version: 0.28.0
I0517 10:00:48.013772   130 exec.cpp:472] Slave exited ... shutting down

Executor stdout... Shutting down (nothing more)

Slave logs (only a part):

I0517 10:00:47.705261    11 slave.cpp:4374] Current disk usage 0.63%. Max allowed age: 6.256098991214965days
I0517 10:00:47.888465     7 slave.cpp:1361] Got assigned task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 for framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:47.889526     7 gc.cpp:83] Unscheduling '/tmp/mesos/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000' from gc
I0517 10:00:47.889643    10 gc.cpp:83] Unscheduling '/tmp/mesos/meta/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000' from gc
I0517 10:00:47.889832     7 slave.cpp:1480] Launching task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 for framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:47.890251     7 paths.cpp:528] Trying to chown '/tmp/mesos/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000/executors/basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003/runs/01392987-0e53-4175-a019-d7b2ba815287' to user 'root'
I0517 10:00:47.894273     7 slave.cpp:5367] Launching executor basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 of framework c0df712a-510c-47f1-84a5-0644c6393726-0000 with resources cpus(*):0.1; mem(*):32 in work directory '/tmp/mesos/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000/executors/basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003/runs/01392987-0e53-4175-a019-d7b2ba815287'
I0517 10:00:47.894803    11 docker.cpp:1009] No container info found, skipping launch
I0517 10:00:47.894989     7 slave.cpp:1698] Queuing task 'basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003' for executor 'basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003' of framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:47.895226     6 containerizer.cpp:666] Starting container '01392987-0e53-4175-a019-d7b2ba815287' for executor 'basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003' of framework 'c0df712a-510c-47f1-84a5-0644c6393726-0000'
I0517 10:00:47.898272    13 launcher.cpp:147] Forked child with pid '124' for container '01392987-0e53-4175-a019-d7b2ba815287'
I0517 10:00:47.898432    13 containerizer.cpp:1118] Checkpointing executor's forked pid 124 to '/tmp/mesos/meta/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000/executors/basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003/runs/01392987-0e53-4175-a019-d7b2ba815287/pids/forked.pid'
I0517 10:00:49.904361     6 slave.cpp:1891] Asked to kill task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 of framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:49.904460     6 slave.cpp:3002] Handling status update TASK_KILLED (UUID: 10164116-b7eb-480f-841e-830f9301e174) for task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 of framework c0df712a-510c-47f1-84a5-0644c6393726-0000 from @0.0.0.0:0
I0517 10:00:49.905267     7 status_update_manager.cpp:320] Received status update TASK_KILLED (UUID: 10164116-b7eb-480f-841e-830f9301e174) for task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 of framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:49.905645     7 status_update_manager.cpp:824] Checkpointing UPDATE for status update TASK_KILLED (UUID: 10164116-b7eb-480f-841e-830f9301e174) for task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 of framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:49.984380    11 slave.cpp:3400] Forwarding the update TASK_KILLED (UUID: 10164116-b7eb-480f-841e-830f9301e174) for task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 of framework c0df712a-510c-47f1-84a5-0644c6393726-0000 to master@MASTER_IP:5051
I0517 10:00:50.013109     7 status_update_manager.cpp:392] Received status update acknowledgement (UUID: 10164116-b7eb-480f-841e-830f9301e174) for task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 of framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:50.013226     7 status_update_manager.cpp:824] Checkpointing ACK for status update TASK_KILLED (UUID: 10164116-b7eb-480f-841e-830f9301e174) for task basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003 of framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:53.040210     7 containerizer.cpp:1608] Executor for container '01392987-0e53-4175-a019-d7b2ba815287' has exited
I0517 10:00:53.040279     7 containerizer.cpp:1392] Destroying container '01392987-0e53-4175-a019-d7b2ba815287'
I0517 10:00:53.042726    11 provisioner.cpp:306] Ignoring destroy request for unknown container 01392987-0e53-4175-a019-d7b2ba815287
I0517 10:00:53.042866     8 slave.cpp:3886] Executor 'basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003' of framework c0df712a-510c-47f1-84a5-0644c6393726-0000 terminated with signal Killed
I0517 10:00:53.042927     8 slave.cpp:3990] Cleaning up executor 'basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003' of framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:53.043233     9 gc.cpp:55] Scheduling '/tmp/mesos/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000/executors/basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003/runs/01392987-0e53-4175-a019-d7b2ba815287' for gc 6.99999950039407days in the future
I0517 10:00:53.043305     9 gc.cpp:55] Scheduling '/tmp/mesos/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000/executors/basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003' for gc 6.99999949956148days in the future
I0517 10:00:53.043344     9 gc.cpp:55] Scheduling '/tmp/mesos/meta/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000/executors/basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003/runs/01392987-0e53-4175-a019-d7b2ba815287' for gc 6.99999949898074days in the future
I0517 10:00:53.043372     8 slave.cpp:4078] Cleaning up framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:53.043376     9 gc.cpp:55] Scheduling '/tmp/mesos/meta/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000/executors/basic-0.3a8b8882-1c16-11e6-89e4-0242ac110003' for gc 6.99999949824296days in the future
I0517 10:00:53.043486    12 status_update_manager.cpp:282] Closing status update streams for framework c0df712a-510c-47f1-84a5-0644c6393726-0000
I0517 10:00:53.043511     9 gc.cpp:55] Scheduling '/tmp/mesos/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000' for gc 6.99999949670222days in the future
I0517 10:00:53.043563     9 gc.cpp:55] Scheduling '/tmp/mesos/meta/slaves/c0df712a-510c-47f1-84a5-0644c6393726-S0/frameworks/c0df712a-510c-47f1-84a5-0644c6393726-0000' for gc 6.99999949613333days in the future

I can probably give more information, if it isn't enough. Thanks for your help !

janisz
  • 6,292
  • 4
  • 37
  • 70
Sunder
  • 503
  • 1
  • 5
  • 17

0 Answers0