Same physical core different NUMA node

Question

Following the steps of identifying which logical cores share the same physical core on my AMD Opteron(TM) Processor 6234, I found out that 2 logical cores sharing the same physical core are not in the same NUMA node.

My steps were to do lscpu in order to get the NUMA nodes related core:

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                24
On-line CPU(s) list:   0-23
Thread(s) per core:    2
Core(s) per socket:    6
Socket(s):             2
NUMA node(s):          4
Vendor ID:             AuthenticAMD
CPU family:            21
Model:                 1
Model name:            AMD Opteron(TM) Processor 6234
Stepping:              2
CPU MHz:               1400.000
CPU max MHz:           2400.0000
CPU min MHz:           1400.0000
BogoMIPS:              4799.98
Virtualization:        AMD-V
L1d cache:             16K
L1i cache:             64K
L2 cache:              2048K
L3 cache:              6144K
NUMA node0 CPU(s):     0-5
NUMA node1 CPU(s):     6-11
NUMA node2 CPU(s):     12-17
NUMA node3 CPU(s):     18-23

Then cat /proc/cpuinfo | grep "core id" in order to group the cores:

core id         : 0
core id         : 1
core id         : 2
core id         : 3
core id         : 4
core id         : 5
core id         : 0
core id         : 1
core id         : 2
core id         : 3
core id         : 4
core id         : 5
core id         : 0
core id         : 1
core id         : 2
core id         : 3
core id         : 4
core id         : 5
core id         : 0
core id         : 1
core id         : 2
core id         : 3
core id         : 4
core id         : 5

And finally cat /proc/cpuinfo | grep "physical id" in order to get the cores sharing the same core id and the same physical CPU:

physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 0
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1
physical id     : 1

From that I conclude that core 0 and 6 share the same physical core, however core 0 is in NUMA node 0, and core 6 is in NUMA node 1.

I find it really disturbing, is that possible?

Thank you in advance for your help.

Answer 1

For Opteron 6234, within each package there are 2 dies where each die is a NUMA node and each die has 6 cores. There's no SMT (hyper-threading) so there's only one CPU per core.

What you'd expect is:

 CPU_number = package:die_in_package:core_in_die:CPU_in_core, NUMA node
 0 = 0:0:0:0, NUMA node 0
 1 = 0:0:1:0, NUMA node 0
 2 = 0:0:2:0, NUMA node 0
 3 = 0:0:3:0, NUMA node 0
 4 = 0:0:4:0, NUMA node 0
 5 = 0:0:5:0, NUMA node 0
 6 = 0:1:0:0, NUMA node 1
 7 = 0:1:1:0, NUMA node 1
 8 = 0:1:2:0, NUMA node 1
 9 = 0:1:3:0, NUMA node 1
10 = 0:1:4:0, NUMA node 1
11 = 0:1:5:0, NUMA node 1
12 = 1:0:0:0, NUMA node 1
13 = 1:0:1:0, NUMA node 2
14 = 1:0:2:0, NUMA node 2
15 = 1:0:3:0, NUMA node 2
16 = 1:0:4:0, NUMA node 2
17 = 1:0:5:0, NUMA node 2
18 = 1:1:0:0, NUMA node 3
19 = 1:1:1:0, NUMA node 3
20 = 1:1:2:0, NUMA node 3
21 = 1:1:3:0, NUMA node 3
22 = 1:1:4:0, NUMA node 3
24 = 1:1:5:0, NUMA node 3

Now look at CPU number 0 and CPU number 6. They both have the same package number, the same NUMA node and the same "CPU in die"; but they have different die numbers. None of the information you've got mentions "die numbers", and that's why you're confused.