3

I'm running a CPU and memory heavy application for 3D reconstruction (www.3dsurvey.si if relevant) on a AMD opteron server (4CPUs, 12cores each, 128GB RAM). I noticed that the CPU is not utilized as expected due to (what I believe) page faults. We get 100k+ page faults per second when processing, while memory is used only around 50%. Average processor utilization we get is around 10%-20%. I also tried to disable pagefile (which is unadvised) with no improvement and same rate of page faults. Software normally uses all processors on regular PC.

To make question applicable to others

  • Why do we get page faults with plenty of RAM still available?
  • How to decrease number of page faults?
mitjap
  • 133
  • 1
  • 5

1 Answers1

8

Page faults can be divided into major and minor faults

Major page faults happen when your program, or its data, was swapped out to disk and now need to be swapped-in from disk. These faults are marked "major" because swapping out/in to/from disk is very slow compared to CPU speed. As you have plenty of free RAM (about 50%), and disabling swapping entirely did not bring any performance back, I think your problem is not related to major faults.

Minor page faults happen when the CPU is trying to access a virtual memory address which is not in its small, fast TLB cache and, as results, it has to lookup a larger (and slower) mapping table stored in know DRAM address. A large amount of minor page faults are expected when run a program sporadically and/or when accessing large amount of memory. This problem can be exacerbated by a multi-socket NUMA topology (the same used by your Opteron) when used with non-NUMA aware programs.

If your program is not NUMA-aware, minor page faults can be the source of yours performance problems. To have a rough idea if this is the case, try to run the program on a single-socket machine (or disable all but one socket on your server) and check if CPU usage is higher than expected.

Anyway, only the software house producing the software (or someone very experienced with your specific program) can completely answer to your question.

shodanshok
  • 47,711
  • 7
  • 111
  • 180
  • True, I was approaching the problem using Unix terminology, but the basic concepts are the same. Feel free to edit my answer. – shodanshok Aug 12 '16 at 14:26
  • Thank you for pointng out the issues with NUMA. I am confident this is the case here. – mitjap Aug 12 '16 at 14:27