2

I have studied the cache and how to utilise it effectively for a few years now. I know all about the hierarchy of caches, how a block of cache is fetched according to cache line, how the prefetcher detects memory access patterns and fetches memory in advance according to it and even how caching works on threads and the pitfalls of caching in multi-threaded programs.

What I have never been able to find out after all this time is how caching works on a computer with multiple concurrently running processes. Over the years, I've realised that my programs are just another process being run alongside other processes in the computer. Even if my program is the only program being run, there will still be the OS running in the background.

With that being said, how do the caches work with multiple processes running concurrently? Are they shared between each of the processes or is the cached memory of one process evicted upon a context switch? Perhaps the answer is a bit of a hybrid of both?

Zachariah the Magnificent
  • 23
  • 5
  • 1
    Data caches are orthogonal to processes (that are just a software concept) and are not usually flushed. Processes can mess up each other cache lines, though empasse situations are rare. Some caches, like the TLBs, are partially flushed on context switches due to the change of page tables. The BTB is a cache in practice and that can also be flushed automatically (IBRS) when entering a particular mode (SMM, SGX), when a predictor mode change (still IBRS) or on demand (IBPB). – Margaret Bloom Mar 16 '18 at 14:14
  • Sounds like you're asking about [address translation](https://en.wikipedia.org/wiki/CPU_cache#Address_translation). I voted to close the question as it's too broad. – Hadi Brais Mar 16 '18 at 16:47

2 Answers2

1

There are few scenarios, lets pick one. In this scenario, caches are accessed with physical addresses.

All the multiple processes (P1, P2...Pn) that are executing in parallel, operate on virtual addresses. We can have the TLB (which holds virtual to physical translation) to flush its entries on a context switch. All the processes can have the same number of virtual pages. But at a given time only few of them are refereed by a process. Therefore you can keep these most used pages in physical memory and the rest in the hard disk. This applies for all the processes currently active.

When process P1 is currently running, when data is required to be fetched from memory, the process is similar to how it is done as if there is only one process. One thing to be note here that when a page fault happens for process P1, if the page that is to be replaced in the physical memory belongs to another process, then that process's page table need to be updated to reflect this.

If you examine the context of physical pages, it can have pages from multiple processes. This fine as page tables of each process will know what virtual page is in which physical location.

Isuru H
  • 1,191
  • 1
  • 8
  • 11
1

Most CPUs are designed with caches that cache based on physical address, so they can still be hot after a context switch, even if a TLB invalidation requires a a page walk to find the right physical page for a virtual address.

If a process migrates to another CPU core, private L1 and L2 will be cold, but shared L3 will still be hot.

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847