0

In USE Method: Linux Performance Checklist it is mentioned that

The goal is a measure of memory capacity saturation - the degree to which a process is driving the system beyond its ability (and causing paging/swapping). [...] Another metric that may serve a similar goal is minor-fault rate by process, which could be watched from /proc/PID/stat.

I'm not sure I understand what minor-faults have to do with memory saturation.

Quoting wikipedia for reference

If the page is loaded in memory at the time the fault is generated, but is not marked in the memory management unit as being loaded in memory, then it is called a minor or soft page fault.

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
jack malkovick
  • 503
  • 2
  • 14

1 Answers1

2

I think what the book is referring to is the following OS behaviour that could make soft page faults increase with memory pressure. But there are other reasons for soft page faults (allocating new pages with mmap(MAP_ANONYMOUS) and then freeing them again; every first-touch of a new page will cost a soft page fault, although fault-around for a group of contiguous pages can reduce that to one fault per N pages for some small N when iterating through a new large allocation.)

When approaching memory pressure limits, Linux (like many other OSes) will un-wire a page in the HW page tables to see if a soft page-fault happens very soon. If no, then it may actually evict that page from memory1.

But if it does soft-pagefault before being evicted, the kernel just has to wire it back in to the page table, having saved a hard page-fault. (And the I/O to write it out in the first place.)

Footnote 1: Writing it to disk if dirty, either in swap space or a file-backed mapping if not anonymous; otherwise just dropping it. The kernel could start this disk I/O while waiting to see if it gets faulted back in; IDK if Linux does this or not.

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
  • What do you mean by "unwire"? Just setting a bit so the fault goes to the OS so it can know that page is still needed? – Noah Feb 14 '21 at 08:30
  • 1
    @Noah: Yes, setting the PTE to "invalid" (not present) and running an `invlpg` (and perhaps a TLB shootdown to invalidate it on other cores). In Unix (and in general), `mmap` logically maps memory to a file (or anonymous backing by swap space), but it's up to the OS whether or not that logical mapping is currently "wired up" in the hardware page tables. "wire" is convenient terminology for that, used by at least some OS. In Linux, memory allocation is usually lazy; `mmap` *doesn't* wire up a new page by default unless you use `MAP_POPULATE`. So ready-to-soft-fault state is not unique. – Peter Cordes Feb 14 '21 at 08:57
  • Do you have any documentation on "fault-around". I'm having trouble finding more than a [less than descriptive lwn post](https://lwn.net/Articles/588802/) on the keyword? Is it like ```MADV_SEQUENTIAL``` + ```MADV_WILLNEED``` i.e fault on page P will create mapping for [P, P + N]? – Noah Feb 14 '21 at 21:10
  • 1
    @Noah: Yes, exactly. Wire up surrounding (available?) pages when you fault on one page. `MADV_SEQUENTIAL` increases the tendency to fault-around, or increases the size of the region, but I think it's already done by default, especially for file-backed mappings. (e.g. reading a file with mmap when the kernel notices a sequential access pattern, which can maybe even happen for code in an executable or library.) – Peter Cordes Feb 14 '21 at 23:34