0

Hello I've been trying to experience cache miss and hits in Linux. To do so, I've done a program in C, where I mesure the time in CPU cycle to do the instruction printf(). The first part mesure the time needed for a miss and the second one for a hit. Here is the given program :

#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sched.h>
#include <sys/types.h> 
#include <unistd.h> 
#include <signal.h>

uint64_t rdtsc() {
  uint64_t a, d;
  asm volatile ("mfence");
  asm volatile ("rdtsc" : "=a" (a), "=d" (d));
  a = (d<<32) | a;
  asm volatile ("mfence");
  return a;
}

int main(int argc, char** argv)
{
    size_t time = rdtsc();
    printf("Hey ");
    size_t delta1 = rdtsc() - time;
    printf("delta: %zu\n", delta1);

    size_t time2 = rdtsc();
    printf("Hey ");
    size_t delta2 = rdtsc() - time2;
    printf("delta: %zu\n", delta2);
    sleep(100);
}

Now I would like to show that two processes (two terminals) have cache in commun. So I thought that running this program in two terminals would result in :

Terminal 1:
miss
hit
Terminal 2:
hit 
hit

But now I have something like:

Terminal 1:
miss
hit
Terminal 2:
miss 
hit

Is my understanding incorrect? Or my program wrong?

Scalinsa
  • 1
  • What do you think should be cached here? – Omry Oct 21 '20 at 12:46
  • I thought that the function *printf()* would be cached, so the Terminal 2 wouldn't have to wait as long as Terminal 1 for the first *printf()*. – Scalinsa Oct 21 '20 at 12:58
  • can you print the delta? maybe the 2 terminals are running from 2 different cores. – pio Nov 14 '20 at 09:38
  • `mfence` isn't guaranteed to interact with `rdtsc` at all. (It does on Skylake because of microcode updates to make `mfence` even slower and safe even with NT loads from WC memory, by adding lfence-like semantics to it.) `lfence` is the instruction that's guaranteed to order instruction execution on Intel CPUs. Also, BTW, there are intrinsics for `_mm_lfence()`. If you're on a CPU where `mfence` doesn't have `lfence` semantics built in, you could use both if you want to wait for the store buffer to drain *and* for all earlier instruction to retire (drain the ROB), like a serializing insn. – Peter Cordes Nov 11 '22 at 07:15

1 Answers1

1

Your assumption is somewhat correct.
printf is part of the libc library. If you use dynamic linking, the operating system may optimize memory usage by only loading the library once for all processes using it.
However, there are multiple reasons why I don't expect you to measure any sizable difference:

  1. compared to the difference between a cache hit and cache miss, printf takes an enormous amount of time to complete and there is a lot going on that introduces noise. With just a single measurement, it is very unlikely that you're able to measure that tiny difference.
  2. the actual reason for the first measurement to take longer is likely the lazy binding of the library function printf being resolved by the loader (https://maskray.me/blog/2021-09-19-all-about-procedure-linkage-table) or some other magic happening (buffers being setup, etc.) for the first output.
  3. a lot of libc functions are used by many different processes. If the library is shared, it is likely, that printf may be cached even though you did not use it.

I would suggest to mount a Flush+Reload attack (https://eprint.iacr.org/2013/448.pdf) on printf in one of the terminals and use it in the other terminal. Then, you may see a timing difference.
Note: to find the actual address of printf for the attack, you need to be familiar with dynamic linking and the plt. Just using something like void* addr = printf will probably not work!

Lorenz Hetterich
  • 296
  • 5