6

Consider this scenario: a user process running on a NUMA machine calls mmap to creates a new mapping in the virtual address space. It then uses the memory returned by mmap for its processing (storing its data,...). Now for some reasons, the user process is scheduled to a different NUMA node. Is it possible for the user process to tell the OS to relocate the already mapped memory (to a different NUMA node) while preserving the data?

osgx
  • 90,338
  • 53
  • 357
  • 513
Bao Bui
  • 185
  • 2
  • 6
  • There is mremap for moving allocation and there are libnuma functions to move physical memory: http://man7.org/linux/man-pages/man2/migrate_pages.2.html – osgx Nov 03 '17 at 13:42
  • I'm aware of migrate_pages function and libnuma. But I don't want to relocate all of the pages, just the previously mapped memory done by mmap. – Bao Bui Nov 03 '17 at 13:46
  • With `mbind` you can migrate subset of pages: http://man7.org/linux/man-pages/man2/mbind.2.html Example:https://stackoverflow.com/a/41385103 kernel doc https://www.kernel.org/doc/Documentation/vm/page_migration "*allows a process to manually relocate the node on which its pages are located through the MF_MOVE and MF_MOVE_ALL options while setting a new memory policy via mbind()*" – osgx Nov 03 '17 at 13:46

1 Answers1

4

Migrating of physical memory is possible with migrate_pages call from libnuma (-lnuma): http://man7.org/linux/man-pages/man2/migrate_pages.2.html

 long migrate_pages(int pid, unsigned long maxnode,
                      const unsigned long *old_nodes,
                      const unsigned long *new_nodes);

Link with -lnuma.

migrate_pages() attempts to move all pages of the process pid that are in memory nodes old_nodes to the memory nodes in new_nodes. Pages not located in any node in old_nodes will not be migrated. As far as possible, the kernel maintains the relative topology relationship inside old_nodes during the migration to new_nodes.

There is also tool migratepages in numactl package to migrate all pages of pid: http://man7.org/linux/man-pages/man8/migratepages.8.html

You also can change memory policy with set_mempolicy: http://man7.org/linux/man-pages/man2/set_mempolicy.2.html

mbind syscall can be used to migrate subset of pages to some NUMA node:

https://www.kernel.org/doc/Documentation/vm/page_migration

...allows a process to manually relocate the node on which its pages are located through the MF_MOVE and MF_MOVE_ALL options while setting a new memory policy via mbind()

http://man7.org/linux/man-pages/man2/mbind.2.html

  If MPOL_MF_MOVE is specified in flags, then the kernel will attempt
   to move all the existing pages in the memory range so that they
   follow the policy.  Pages that are shared with other processes will
   not be moved.  If MPOL_MF_STRICT is also specified, then the call
   will fail with the error EIO if some pages could not be moved.
osgx
  • 90,338
  • 53
  • 357
  • 513