0

server: i have servers with 2 intel 10 cores cpus or 8 cores. So some has 40 cores, some has 32 cores (enable intel HT)

background: i am running our application, which will isolate cpus, currently, i isolate the last 32 cores (core 8-39) for that application. 4 cores (core 4-7) for other use(normally, it will used 50% sys cpu). And i want to assign core 0-3 for system IRQ usage. since currently, if i run the application, system response is very slow, i think some of irq requests have been disputed to core 4-7, that cause low response. do you think if that is possible just use 4 cores to handle system irq?

yixuan
  • 3
  • 6

1 Answers1

0

If you have more then one socket ("stone") that means you have NUMA system. Here is a link to get more info https://en.wikipedia.org/wiki/Non-uniform_memory_access

Try to use CPUs on the same socket. Below I will explain why and how to do that

  1. Determine what exactly CPU ids located on each socket.

    % numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 node 0 size: 24565 MB node 0 free: 2069 MB node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 node 1 size: 24575 MB node 1 free: 1806 MB node distances: node 0 1 0: 10 20 1: 20 10

Here "node" means "socket" (stone). So 0,2,4,6 CPUs are located on the same node. And it makes sense to move all IRQs into one node to use L3 cache for set of CPUs.

  1. Isolate all CPUs except 0,2,4,6.

Need to add argument to start Linux kernel isolcpus= cpu_number [, cpu_number ,...]

for example

isolcpus=1,3,5,7-31
  1. Control what IRQs are running on what CPUs

cat /proc/interrupts

  1. Start your application with numactl command to aligne to CPUs and Memory. (Here need to understand what NUMA and aligned is. Please follow the link at the beginning of the article)

numactl [--membind=nodes] [--cpunodebind=nodes]

  1. Your question is much bigger than I mentioned here.

If you see the system is slow need to understand bottleneck. Try to gather raw info with top, vmstat, iostat to find out the point of weakness.

Provide some stat of your system and I will help you to turn it up right way.

Moonray
  • 41
  • 4