3

I writing NUMA-aaware cache for large objects (matrices of doubles) for 4 socket server. I observe that intersocket communication is the bottleneck for my application. Hence, I want threads on different sockets to have separate matrix caches. I have bounded threads to specific physical processors and now I need to make threads select correct cache.

Suppose cache is defined in the following way:

matrix_cache_t *cache[SOCKETS_LIMIT];

I need each thread to know its socket id and select correct cache, e.g. cache[0], cache[1], cache[2] or cache[3].

I am writing the application in C using OpenMP and it is supposed to run on both Windows and Linux.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
cdome
  • 31
  • 2
  • What platform are you in? – merlin2011 Aug 18 '14 at 22:39
  • x86 and x86-64, only recent hardware support is required, as application already depends on SSE4 instruction set – cdome Aug 18 '14 at 23:04
  • consider [hwloc](http://www.open-mpi.org/projects/hwloc/]) and read this https://stackoverflow.com/questions/14720642/openmp-parallel-for-region-thread-affinity – Z boson Aug 19 '14 at 07:23
  • and see this https://stackoverflow.com/questions/24957781/openmp-splitting-loop-based-on-numa/24959025#24959025 – Z boson Aug 19 '14 at 07:25

1 Answers1

0

Under Windows you can use the GetLogicalProcessorInformationEx() API using either the RelationNumaNode or RelationProcessorPackage relation. It gives you the bits of all the processors in the respective relationship, which correspond to the affinity bits used to tie a thread to a processor.

Under Linux you can use sched_getcpu

#include <stdio.h>
#include <sched.h>

int sched_getcpu();

int main()
{
    (void) printf("cpu %d\n", sched_getcpu());
}

Socket id can be found in /proc/cpuinfo or /sys/devices/system/cpu/cpu0/topology/physical_package_id

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
Alexander Gessler
  • 45,603
  • 7
  • 82
  • 122
  • @Fredrik Widlund - Thanks for adding the Linux part. Made it CW. – Alexander Gessler Aug 18 '14 at 22:48
  • @Fredrik Widlund - very useful. Do you know how to get from sched_getcpu() to socket id? Thank you – cdome Aug 18 '14 at 23:02
  • @cdome I suspect you'd have to parse `/proc/cpuinfo` for that. Mind that the format of this file is architecture-dependent. –  Aug 18 '14 at 23:31
  • @duskwuff - I agree, both cpuinfo and /sys/devices/system/cpu/... are AFAIK platform dependent. (btw, just now reached enough rep to comment here, which is why I answered in the post) – Fredrik Widlund Aug 19 '14 at 00:33