Same IDTR value for each cpu

Question

I wrote a kernel module to print IDTR value to see if each CPU (hyper-threads) running on each core have their own separate IDT or share the same one for a particular Core. But, I am getting same base and limit value for IDTR on each CPU, which makes it seem like that there is a global IDT that is shared by all the CPUs.

To run on different CPUs, I wrote a script that inserts my module using a particular CPU using 'taskset'.

This is my module:

#include <linux/init.h>
#include <linux/module.h>

struct idt {
    u16 length;
        u64    base;
} __attribute__((packed)) idt;

void my_store_idt(struct idt *idt)
{
    asm volatile ("sidt %0" : "=m"(*idt));
}

static int __init mod_init(void) {
        my_store_idt(&idt);
        printk("cpu = %d, idtr.length = %u, idtr.base =  %llu\n"
                ,get_cpu(), idt.length, idt.base);

        return 0;
}

static void __exit mod_exit(void)
{

        printk("byeee\n");
}

module_init(mod_init);
module_exit(mod_exit);
MODULE_LICENSE("GPL");

And this is the output I get:

[Sep15 00:52] cpu = 0, idtr.length = 4095, idtr.base =  18446741874686296064
[  +0.001625] byeee
[  +0.016648] cpu = 1, idtr.length = 4095, idtr.base =  18446741874686296064
[  +0.001234] byeee
[  +0.012360] cpu = 2, idtr.length = 4095, idtr.base =  18446741874686296064
[  +0.001140] byeee
[  +0.007094] cpu = 3, idtr.length = 4095, idtr.base =  18446741874686296064
[  +0.001182] byeee
[  +0.010665] cpu = 4, idtr.length = 4095, idtr.base =  18446741874686296064
[  +0.001136] byeee
[  +0.007931] cpu = 5, idtr.length = 4095, idtr.base =  18446741874686296064
[  +0.001141] byeee
[  +0.012969] cpu = 6, idtr.length = 4095, idtr.base =  18446741874686296064
[  +0.001189] byeee
[  +0.019079] cpu = 7, idtr.length = 4095, idtr.base =  18446741874686296064
[  +0.001226] byeee

This is the scrpt I used to run it on different CPUs:

 #!/bin/bash

 for i in {0..7} 
  do

      taskset -c $i insmod read_idtr.ko

      rmmod read_idtr 
  done

I have read that each processor has its own IDT. So, either it should show different IDTR value for each CPU or for atleast each Core. Why I am getting the same for each CPU?

Please: do not post images of texts. You can simply copy/paste from console. It really help us — Giacomo Catenazzi, Sep 15 '22 at 13:24
I am new to 'stack overflow' and it is not allowing me to embed image directly until I earn 10 reputation points. — Syed Tayyab Farooq, Sep 15 '22 at 13:32
Good. No one has asked you to add an image. We do not want you to post an image. We want you to copy & paste the relevant text *as text*. — John Bollinger, Sep 15 '22 at 13:37
Also, "am I doing something wrong?" is a rather vague question. What makes you think something is wrong? What aspects of the results you observe (but we don't yet, because you've tried to convey it as an image) surprise you? — John Bollinger, Sep 15 '22 at 13:43
Thanks for the suggestions. I have incorporated them in my post. — Syed Tayyab Farooq, Sep 15 '22 at 14:32
For Linux x86, the IDT is actually stored in the statically allocated (not per-cpu), page-aligned array variable `idt_table` defined in "arch/x86/kernel/idt.c". It is mapped to a fixed virtual address in the CPU entry area by the `idt_map_in_cea()` function. — Ian Abbott, Sep 15 '22 at 15:24
So, does this mean that there is only a single IDT or that we are only allowed to access one? And, if there are multiple IDTs but we can not access them normally, then is there any workaround to access them? — Syed Tayyab Farooq, Sep 19 '22 at 10:49

John Bollinger · Answer 1 · 2022-09-15T14:07:18.547

I wrote a kernel module to print IDTR value to see if each CPU (hyper-threads) running on each core have their own separate IDT or share the same one for a particular Core. But, I am getting same base and limit value for IDTR on each CPU, which makes it seem like that there is a global IDT that is shared by all the CPUs.

And? That seems like a valid and eminently reasonable result to me. Why would you expect a multiprocessing OS kernel to establish different interrupt vectors for different cores? And if the cores are all to have the same interrupt vectors, then why give each one its own copy of the IDT? Better all around in that case for all execution units to share the same IDT.

In any case, as the basis for your experiment, you already hypothesized that different virtual CPUs associated with the same core might have the same IDT, so I don't see what the problem is. Of course, your experiment does not show whether the IDTRs are per-core or per-hyperthread, but its design cannot distinguish between those alternatives in the per-core case anyway, and it might not distinguish between them in the per-hyperthread case. That is, when you see different virtual CPUs having the same IDT, the expirement does not tell you whether that's because they are incapable of having different ones or because the system just set them up that way.

Am I doing something wrong?

If anything is wrong with your code then I don't see it, but I suspect that some of your assumptions are wrong. Also, reading specs would probably be a better approach to answering your question than is performing the experiment you have devised.

If there is only a single and global IDT for all the cores then the max numbers of IRQ vectors that can be allocated are only 224 (I read this in a post: https://stackoverflow.com/questions/48269741/interrupt-descriptor-table-in-multiprocessor-system), which are not enough for modern applications. Also, I checked number of vectors in /proc/irq and it exceeds 224, which shows that more interrupts are currently assigned than a single IDT can hold. — Syed Tayyab Farooq, Sep 15 '22 at 14:31
Then you can conclude, @SyedTayyabFarooq, that `/proc/irq` is not reporting on an IDT. FWIW, however, "not enough for modern applications" seems specious: on my general-use, multi-core workstation, `/proc/irq` has fewer than 30 entries. — John Bollinger, Sep 15 '22 at 14:48
@SyedTayyabFarooq, it may well be that there are times that different execution units will have different IDTs, but there is no reason at all to suppose that every available IDTR will always specify a distinct IDT, nor even that at any given time, *any* IDTR will specify an IDT distinct from the others. — John Bollinger, Sep 15 '22 at 15:04
But one possibility is that the kernel sets a standard IDT when processing a module insertion (for the CPU processing that insertion). Since that is the only context in which you check an IDTR, that would lead to always observing the same one. I don't know for sure whether that's the kernel's actual behavior, but it seems plausible to me. — John Bollinger, Sep 15 '22 at 15:06

Same IDTR value for each cpu

1 Answers1