0

Would anyone be able to help with info on why I have such a high server load but total CPU usage is under 50% and loads of free RAM?

I'm running 60 instances of ffmpeg on a Ryzen 1700X.

Here's a sample:

top - 23:06:56 up 14 days,  9:18,  1 user,  load average: 37.60, 29.48, 27.77
Tasks: 300 total,   4 running, 296 sleeping,   0 stopped,   0 zombie
%Cpu(s): 49.4 us,  2.9 sy, 24.2 ni, 22.3 id,  0.0 wa,  0.0 hi,  1.2 si,  0.0 st
KiB Mem : 65973428 total, 56648208 free,  6166708 used,  3158512 buff/cache
KiB Swap: 33521660 total, 33521660 free,        0 used. 58950204 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
26716 root      20   0 1785484 109948  15040 S  30.2  0.2   8:05.60 ffmpeg
26466 root      20   0 1785384 110252  15212 S  29.9  0.2   7:55.21 ffmpeg
23944 root      20   0 1784344  95248  14968 S  27.9  0.1  11:42.94 ffmpeg
26714 root      20   0 1786092 110624  15152 S  27.9  0.2   7:41.62 ffmpeg
22213 root      20   0 1784096 109416  15208 S  27.2  0.2  15:58.07 ffmpeg
24430 root      20   0 1785028 109392  15180 S  26.9  0.2  12:51.34 ffmpeg
23199 root      20   0 1783992  95164  15092 S  26.2  0.1  13:50.43 ffmpeg
26220 root      20   0 1785880 110148  15192 S  26.2  0.2   8:00.28 ffmpeg

and here's an mpstat -P ALL output

root@Ubuntu-1604-xenial-64-minimal ~ # mpstat -P ALL
Linux 4.8.0-58-generic (Ubuntu-1604-xenial-64-minimal)  07/27/2017      _x86_64_        (16 CPU)

11:08:49 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
11:08:49 PM  all   35.80   16.81    2.27    0.00    0.00    1.03    0.00    0.00    0.00   44.09
11:08:49 PM    0   34.66   12.75    3.47    0.00    0.00   10.27    0.00    0.00    0.00   38.85
11:08:49 PM    1   31.05   15.86    3.41    0.00    0.00    0.96    0.00    0.00    0.00   48.72
11:08:49 PM    2   38.62   16.16    2.55    0.00    0.00    0.60    0.00    0.00    0.00   42.07
11:08:49 PM    3   33.36   17.36    2.49    0.00    0.00    0.57    0.00    0.00    0.00   46.21
11:08:49 PM    4   39.13   16.26    2.47    0.00    0.00    0.49    0.00    0.00    0.00   41.65
11:08:49 PM    5   33.42   17.36    2.47    0.00    0.00    0.56    0.00    0.00    0.00   46.19
11:08:49 PM    6   39.25   16.22    2.43    0.00    0.00    0.47    0.00    0.00    0.00   41.63
11:08:49 PM    7   33.42   17.23    2.46    0.00    0.00    0.55    0.00    0.00    0.00   46.34
11:08:49 PM    8   38.73   17.16    1.89    0.00    0.00    0.28    0.00    0.00    0.00   41.94
11:08:49 PM    9   33.78   17.71    1.77    0.00    0.00    0.26    0.00    0.00    0.00   46.48
11:08:49 PM   10   38.76   17.25    1.88    0.00    0.00    0.27    0.00    0.00    0.00   41.84
11:08:49 PM   11   33.75   17.77    1.77    0.00    0.00    0.26    0.00    0.00    0.00   46.44
11:08:49 PM   12   38.68   17.15    1.88    0.00    0.00    0.27    0.00    0.00    0.00   42.02
11:08:49 PM   13   33.73   17.70    1.76    0.00    0.00    0.26    0.00    0.00    0.00   46.55
11:08:49 PM   14   38.67   17.18    1.88    0.00    0.00    0.27    0.00    0.00    0.00   41.99
11:08:49 PM   15   33.72   17.73    1.76    0.00    0.00    0.26    0.00    0.00    0.00   46.53

I'm a bit concerned of the load going to 35+, it's an 8 core 16 thread CPU but, hovering around 35 seems a bit high. Total bandwidth in is around 30mbps (30 video streams being encoded to 2 smaller bitrates, hence 60 ffmpeg processes) and bandwidth out is around 100mbps, the server has a 1gbps port.

Does anyone have any idea why the load so high on a high spec system with plenty of spare resources?

Paul
  • 11
  • 1
  • I think you have a misunderstanding of what 'load' means on Linux. Load is basically a count of processes waiting for a CPU to run. You have 16 CPUs, and 60 processes running. 35 processses waiting for CPU is totally reasonable. The amount of RAM you have has nearly nothing to do with the amount of RAM available except in the case you're running out and swap activity increases CPU usage / blocks processes waiting for stuff to swap back in. –  Jul 27 '17 at 21:15

1 Answers1

0

Ryzen 1700X CPU has 16 threads so simultaneously it can run 16 processes maximum. 60 instances of ffmpeg leaves at most 44 waiting for their turn. Since processes also wait for I/O to finish amount other things load average of 37.60 seems ok.

If you want to understand how busy your CPU at a given moment using top, sum percent numbers except for id (idle). Your output:

%Cpu(s): 49.4 us,  2.9 sy, 24.2 ni, 22.3 id,  0.0 wa,  0.0 hi,  1.2 si,  0.0 st

corresponds to 77.7% not 50%.

RLazar
  • 101
  • 3