Why do allocations with numa_alloc_onnode() lead to "The page is not present"?

Question

When I allocate memory on a specific NUMA node using numa_alloc_onnode() like this :

char *ptr;
if ((ptr = (char *) numa_alloc_onnode(1024,1)) == NULL) {
  fprintf(stderr,"Problem in %s line %d allocating memory\n",__FILE__,__LINE__);
  return(1);
}

and then use move_pages() to try and confirm that the memory allocated is indeed on node 1 :

  printf("ptr is on node %d\n",get_node(ptr));

where

// This function returns the NUMA node that a pointer address resides on.

int get_node(void *p)
{
  int status[1];
  void *pa;
  unsigned long a;

  // round p down to the nearest page boundary

  a  = (unsigned long) p;
  a  = a - (a % ((unsigned long) getpagesize()));
  pa = (void *) a;    

  if (move_pages(0,1,&pa,NULL,status,0) != 0) {
    fprintf(stderr,"Problem in %s line %d calling move_pages()\n",__FILE__,__LINE__);
    abort();
  }

  return(status[0]);

}

I get the answer "ptr is on node -2". From errno-base.h I find that 2 is ENOENT, and the move_pages() man page says that a status of -ENOENT in this context means "The page is not present".

If I replace numa_alloc_onnode() with ordinary malloc() it works fine : I get a node number.

Does anyone have any idea what's going on here?

Thanks in advance.

score 5 · Accepted Answer · answered Nov 12 '11 at 01:15

5

numa_alloc_onnode(3) says:

   All numa memory allocation policy only takes effect when a
   page is actually faulted into the address space of a process
   by accessing it. The numa_alloc_* functions take care of this
   automatically.

Does this mean you need to store something into the newly-allocated page before the kernel actually gives you the page?

answered Nov 12 '11 at 01:15

sarnold

102,305
22
181
238

1

You are correct. You beat me by 12 seconds! It turns out that writing to the allocated space (a simple ptr[0] = 0; will do) makes the above code work as expected : it gives me a node number. Reading doesn't count, it seems. It also seems to me that this contradicts the man page : this is a numa_alloc_* function and it wasn't taken care of automatically! I'm on Debian Squeeze, if that's relevant. – Rob_before_edits Nov 12 '11 at 01:17
1

Heh, that bit about 'automatically' confused me too -- but then I remembered it was written by a German Kernel engineer who probably thinks of "setting up the page faulting mechanism automatically". :) – sarnold Nov 12 '11 at 01:22

Why do allocations with numa_alloc_onnode() lead to "The page is not present"?

1 Answers1