6

I'm quite new to Prometheus, so this is possibly a silly question - but here goes.

When I request node_memory_Active_bytes and node_memory_MemTotal_bytes I get:

node_memory_Active_bytes{instance="10.1.4.9:9100",job="node-exporter"}  3257815040

(..and..)

node_memory_MemTotal_bytes{instance="10.1.4.9:9100",job="node-exporter"}    16509550592

However, on the box, if I run free -b, I get the following:

$ free -b
              total        used        free      shared  buff/cache   available
Mem:    16509550592  2264915968  6787731456    59121664  7456903168 14140530688
Swap:             0           0           0

I'm struggling to maps prometheus's view of memory with the actual machine's view. I'm sure I must be doing something wrong, but I have no idea what. The total's match, but the free

Matthew Haworth
  • 426
  • 1
  • 5
  • 17

2 Answers2

24

Equivalent queries for free -b command:

  • total: node_memory_MemTotal_bytes
  • used: node_memory_MemTotal_bytes - node_memory_MemFree_bytes - node_memory_Buffers_bytes - node_memory_Cached_bytes - node_memory_SReclaimable_bytes
  • shared: node_memory_Shmem_bytes
  • free: node_memory_MemFree_bytes
  • buff/cache: node_memory_Buffers_bytes + node_memory_Cached_bytes + node_memory_SReclaimable_bytes
  • available: node_memory_MemAvailable_bytes
Naser Mirzaei
  • 698
  • 7
  • 11
  • 1
    You save my day! This should be the right answer. – Edmund Wang Jul 13 '21 at 05:39
  • The `used` value from `free` correlates with: `node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes` Not as stated by this answer: `node_memory_MemTotal_bytes - node_memory_MemFree_bytes - node_memory_Buffers_bytes - node_memory_Cached_bytes - node_memory_SReclaimable_bytes` – StudioLE Jul 29 '23 at 06:13
  • The man page even states `Used or unavailable memory (calculated as total - available)` https://www.man7.org/linux/man-pages/man1/free.1.html – StudioLE Jul 29 '23 at 06:15
9

I think you're confusing "active" memory with "used" memory, aren't you? They're different things.

See more info about this here.

To get "used" memory with Prometheus Node Exporter calculate:

node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes
Marcelo Ávila de Oliveira
  • 19,950
  • 3
  • 39
  • 50
  • 1
    Ahh! Makes sense, it's still a bit out though. `node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes = 2394025984` `used = 2282270720` Any reason for that? This happens consistently over a fair sample of time. – Matthew Haworth Jan 14 '20 at 18:30
  • Uhm... I don't know :-( – Marcelo Ávila de Oliveira Jan 14 '20 at 19:43
  • The discrepancy is possibly due to the estimated nature of `available`. From the man page: Estimation of how much memory is available for starting new applications, without swapping. Unlike the data provided by the cache or free fields, this field takes into account page cache and also that not all reclaimable memory slabs will be reclaimed due to items being in use (MemAvailable in /proc/meminfo, available on kernels 3.14, emulated on kernels 2.6.27+, otherwise the same as free) – StudioLE Jul 29 '23 at 06:56