0

I’m currently deploying a kvm/qemu virtual machine with mesos/marathon. In marathon, I’m using the built in mesos command executor and running the script. 

virsh start centos7.0; while true; do echo 'centos 7.0 guest is running'; sleep 5; done

Note the while loop is there only to keep the task running. My issue is that I cannot get mesos to monitor the resource usage of the virtual machine.

When marathon deploys this task on a mesos-agent, it is creating a container that uses the memory and cpu cgroups.
/sys/fs/cgroup/cpu/mesos/31b48dc3-6f09-4b5a-8964-b82d711bb895
/sys/fs/cgroup/memory/mesos/31b48dc3-6f09-4b5a-8964-b82d711bb895

When the virtual machine is being kicked off, the virsh start command is sending a request to libvirtd. Libvirtd then reads the guest.xml file located in /etc/libvirt/qemu/ and then sends a request to the qemu/kvm driver to deploy it.

In my guest.xml file I’m using a custom partition cgroup slice to monitor my virtual machine usage.

https://libvirt.org/cgroups.html
(for each cgroup)
/sys/fs/cgroup/???/vmHolder.slice/machine-qemu\x2d1\x2dcentos7.0\x2dclone.scope

What I have tried. I tried deleting my memory / cpu cgroup from this slice by doing cgdelete -r cpu,memory:vmHolder.slice

and then adding my qemu guest process to the mesos controllers cgclassify -g cpu,memory:mesos/31b48dc3-6f09-4b5a-8964-b82d711bb895 GUEST-PID

When I run the command cat /proc/5531/cgroup

11:perf_event:/vmHolder.slice/machine-qemu\x2d1\x2dcentos7.0\x2dclone.scope
10:pids:/
9:devices:/vmHolder.slice/machine-qemu\x2d1\x2dcentos7.0\x2dclone.scope
8:cpuset:/vmHolder.slice/machine-qemu\x2d1\x2dcentos7.0\x2dclone.scope/emulator
7:net_prio,net_cls:/vmHolder.slice/machine-qemu\x2d1\x2dcentos7.0\x2dclone.scope
6:freezer:/vmHolder.slice/machine-qemu\x2d1\x2dcentos7.0\x2dclone.scope
5:blkio:/vmHolder.slice/machine-qemu\x2d1\x2dcentos7.0\x2dclone.scope
4:hugetlb:/
3:cpuacct,cpu:/mesos/31b48dc3-6f09-4b5a-8964-b82d711bb895
2:memory:/mesos/31b48dc3-6f09-4b5a-8964-b82d711bb895
1:name=systemd:/vmHolder.slice/machine-qemu\x2d1\x2dcentos7.0\x2dclone.scope

It shows that I’m using those controllers, but when I run systemd-cgtop it's not adding the memory usage of the VM. I'm not sure what to do next. Any suggestions?

Jason
  • 23
  • 2
  • Why do you need to monitor resources with Mesos? AFAIK mesos only shows some metrics from enabled isolators but do not interpret them. – janisz Jul 06 '17 at 21:09
  • It’s more of letting mesos know that the virtual machine that’s outside the dedicated resources that it has provided for my task is part of my task and its consuming resources. I can setup a task that requires 8gb of memory and 2 cpus and deploy the virtual machine from it. Mesos with think I’m only using 6mb of memory and .01 cpu, though I’m really using all the resources. – Jason Jul 14 '17 at 16:57

0 Answers0