3

I am working on a quad Operton 6272 system with CentOS installed on it. I suspect there is something wrong with the NUMA configuration.

When I run numactl --hardware I get:

available: 5 nodes (0,2-4,6)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
node 0 size: 32765 MB
node 0 free: 31145 MB
node 2 cpus: 16 17 18 19 20 21 22 23
node 2 size: 16384 MB
node 2 free: 15501 MB
node 3 cpus: 24 25 26 27 28 29 30 31 40 41 42 43 44 45 46 47
node 3 size: 16384 MB
node 3 free: 14913 MB
node 4 cpus: 32 33 34 35 36 37 38 39
node 4 size: 32768 MB
node 4 free: 31551 MB
node 6 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
node 6 size: 32752 MB
node 6 free: 31575 MB
node distances:
node   0   2   3   4   6 
  0:  10  16  22  16  16 
  2:  16  10  16  16  16 
  3:  22  16  10  22  22 
  4:  16  16  22  10  16 
  6:  16  16  22  16  10

There are 4 CPU chips so having 5 NUMA nodes makes no sense to me.

Can anyone please tell me where are CPU cores assigned to NUMA nodes?

Alexander Chertov
  • 2,070
  • 13
  • 16
  • 1
    Technically it's possible that there's more than one memory bank per CPU. The relationship between CPUs and NUMA nodes is likely documented in the BIOS extensions (`SRAT`, etc) – Brian Cain Oct 07 '13 at 13:51

1 Answers1

1

Do you have any kernel boot options defined for memory layout? Can you also post dmesg from boot where the numa nodes are listed with memory ranges? Also, will be nice to know kernel version and libnuma version.

cosmoecho
  • 26
  • 1
  • 1
    The situation has been greatly clarified since the time I posted this question. Should have written back here. Basicly one of the memory sticks was put in a "wrong" slot. The solution was to tweak BIOS settings, put the RAM sticks into different slots, reboot and experiment. I no longer have access to that machine, so I better close the question. – Alexander Chertov Jan 16 '14 at 06:01