How to get Mesos Agents Framework Executor Memory

Question

Inside Mesos Web UI I can see memory usage of my Spark executors in a table

Agents -> Framework -> Executors

There is a table listing all executors for my Spark driver and their memory usage is indicated in column Mem (Used / Allocated).

Is there a way to obtain this number directly via a link and if yes how?

For example I can obtain a bunch of Mesos metrics via http://IP/mesos/metrics/snapshot but memory usage of executors is not one of them.

score 0 · Answer 1 · answered Jun 05 '18 at 07:26

0

The memory usage of executors in fact is related with mesos task, means for every task how many memory the executors will consume.

If above is what you need, you can use following rest api to get a json and then parse the memory used from it.

http://mesos_ip:5050/master/tasks

FYI.

answered Jun 05 '18 at 07:26

atline

28,355
16
77
113

thanks for the info. Aggregate over running tasks might have worked (didn't try that) but I found the more direct access to the memory info I was looking for. See my answer. – astro_asz Jun 05 '18 at 12:53

score 0 · Accepted Answer · answered Jun 05 '18 at 12:52

Found the answer myself. For each worker/agent on which executors may run, direct access to memory info is here:

http://IP_of_worker1:5051/slave(1)/monitor/statistics
http://IP_of_worker2:5051/slave(1)/monitor/statistics
etc

The content is in the form of a json and framework_id allows to find the related executors and their memory consumption, cpu usage etc what is given in the table.

How to get Mesos Agents Framework Executor Memory

2 Answers2

Linked