4

I am trying to use mbind() in my C++ code in order to rearrange virtual pages across 4 NUMA domains, unfortunately I am new to this function:

long mbind(void *addr, unsigned long len, int mode, const unsigned long *nodemask,   
              unsigned long maxnode, unsigned flags);

Currently, I have something like this:

mbind(0x0,4611686018424767488,MPOL_BIND,nodemask,maxnode,MPOL_MF_MOVE);

From the specs it's still unclear to me what to put and how to put for nodemask and maxnode.

Himanshu
  • 4,327
  • 16
  • 31
  • 39
user4530988
  • 63
  • 1
  • 7

1 Answers1

3

nodemask is a pointer to a bitmask of the allowed NUMA nodes. The bitmask is an array of unsigned long elements with each array element holding as many bits as the size of unsigned long int on the particular architecture allows. Unless your program is running on a really big NUMA system, a single unsigned long variable should suffice.

maxnode gives the number of significant bits in nodemask. The kernel rounds internally the size to a multiple of sizeof(unsigned long) but only uses maxnode bits.

There are many examples and libraries around that allow you to create and conveniently manipulate bitmasks without having to fiddle with bit operations yourself. You can utilise libnuma. It doesn't allow you to set the MPOL_MF_MOVE policy but includes functions for the creation and manipulation of nodemasks.

A terrible and very unportable Linux pro-tip: The existing macros that deal with the CPU affinity masks, namely CPU_ZERO / CPU_SET / CPU_CLR and the associated data structure cpu_set_t can also be used for NUMA nodemasks. The reason for that is that (1) both are implemented as arrays of unsigned long and (2) there are usually less NUMA nodes than logical CPUs, therefore cpu_set_t should have enough bits to be able to represent all NUMA nodes on the system.

Side note: 4611686018424767488 should probably be suffixed with LL.

Hristo Iliev
  • 72,659
  • 12
  • 135
  • 186
  • now i'm suffering with following: ... CPU_ZERO(&cpu_set); CPU_SET(0, &cpu_set); ... CPU_SET(7, &cpu_set); sched_setaffinity(0, sizeof(cpu_set), &cpu_set); nodemask = 8; //ret = set_mempolicy(MPOL_BIND, &nodemask, sizeof(nodemask)*8); ret = mbind(0, 12884901888, MPOL_BIND, &nodemask, sizeof(unsigned long), MPOL_MF_MOVE); this works with set_mempolicy (commented line), but doesn't work with mbind. could u pls help here? tnx! – user4530988 Oct 12 '15 at 01:51