0

I am using NetBSD 5.1 for x86 systems. While studying some driver related code, I see that we use splraise and spllower to block or allow interrupts. I searched some of the mechanisms on internet to understand how these mechanisms work in reality. Did not get any real info on that.

When I disassembled I got the mechanism but still do not understand how all these assembly instruction yield me the result. I know x86 instruction individually, but not how the whole stuff works in its entirety.

Need your help in understanding its principles for x86 system. I understand that we need to disable Interrupt Enable (IE) bit, but this assembly seems to be doing more than just this work. Need help.

  (gdb) x/50i splraise
   0xc0100d40:  mov    0x4(%esp),%edx
   0xc0100d44:  mov    %fs:0x214,%eax
   0xc0100d4a:  cmp    %edx,%eax
   0xc0100d4c:  ja     0xc0100d55
   0xc0100d4e:  mov    %edx,%fs:0x214
   0xc0100d55:  ret
   0xc0100d56:  lea    0x0(%esi),%esi
   0xc0100d59:  lea    0x0(%edi,%eiz,1),%edi
   (gdb) p spllower
   $38 = {<text variable, no debug info>} 0xc0100d60
   0xc0100d60:  mov    0x4(%esp),%ecx
   0xc0100d64:  mov    %fs:0x214,%edx
   0xc0100d6b:  cmp    %edx,%ecx
   0xc0100d6d:  push   %ebx
   0xc0100d6e:  jae,pn 0xc0100d8f
   0xc0100d71:  mov    %fs:0x210,%eax
   0xc0100d77:  test   %eax,%fs:0x244(,%ecx,4)
   0xc0100d7f:  mov    %eax,%ebx
   0xc0100d81:  jne,pn 0xc0100d91
   0xc0100d84:  cmpxchg8b %fs:0x210
   0xc0100d8c:  jne,pn 0xc0100d71
   0xc0100d8f:  pop    %ebx
   0xc0100d90:  ret
   0xc0100d91:  pop    %ebx
   0xc0100d92:  jmp    0xc0100df0
   0xc0100d97:  mov    %esi,%esi
   0xc0100d99:  lea    0x0(%edi,%eiz,1),%edi
   0xc0100da0:  mov    0x4(%esp),%ecx
   0xc0100da4:  mov    %fs:0x214,%edx
   0xc0100dab:  cmp    %edx,%ecx
   0xc0100dad:  push   %ebx
   0xc0100dae:  jae,pn 0xc0100dcf
   0xc0100db1:  mov    %fs:0x210,%eax
   0xc0100db7:  test   %eax,%fs:0x244(,%ecx,4)
   0xc0100dbf:  mov    %eax,%ebx
   0xc0100dc1:  jne,pn 0xc0100dd1
   0xc0100dc4:  cmpxchg8b %fs:0x210
   0xc0100dcc:  jne,pn 0xc0100db1
   0xc0100dcf:  pop    %ebx
   0xc0100dd0:  ret
   0xc0100dd1:  pop    %ebx
   0xc0100dd2:  jmp    0xc0100df0
   0xc0100dd7:  mov    %esi,%esi
   0xc0100dd9:  lea    0x0(%edi,%eiz,1),%edi
   0xc0100de0:  nop
   0xc0100de1:  jmp    0xc0100df0

The code seems to be using a helper function cx8_spllower starting at address 0xc0100da0.

Alexey Frunze
  • 61,140
  • 12
  • 83
  • 180
ultimate cause
  • 2,264
  • 4
  • 27
  • 44
  • 1
    [`cli`](https://www.felixcloutier.com/x86/cli) clears IF (so this core won't service interrupts), `sti` sets IF again, re-enabling interrupts. – Peter Cordes Feb 06 '19 at 11:09
  • I'm not sure what this code is doing, but the function names sound more like semaphore lock/unlock. Is `spl` short for spinlock? Not sure why it would be using `cmpxchg8b %fs:0x210`. I guess that's on thread-local storage, that's what `%fs` is normally used for, and that's why it isn't using the `lock` prefix. (i.e. it provides atomicity wrt. other accesses from the same core, because interrupts can't happen in the middle of one instruction). – Peter Cordes Feb 06 '19 at 11:14
  • 1
    @PeterCordes : On FreeBSD in this context SPL doesn't mean spinlock. It means [S]ystem interrupt [P]riority [L]evel. – Michael Petch Feb 06 '19 at 18:57

1 Answers1

2

Clearing [E|R]FLAGS.IE with e.g. CLI disables all (maskable) interrupts on a CPU. For a number of reasons it may be undesirable (e.g. you want to allow some or, perhaps, you don't want overheads of virtualizing CLI in a VM).

Another way of achieving the goal is to tell to the interrupt controller (either the old 8259 PIC or the Pentium APIC/IOAPIC) that you don't want to service interrupts whose priority is below a certain level. For that you need to communicate with the controller, which itself may incur additional overhead (talking to both real and virtual hardware is slow).

A variant of the latter is to keep the current interrupt level/priority in a variable and let interrupts come in, but only really service those whose level/priority isn't lower than the current one in that variable. Those unserviced will be marked as pending and will get fully serviced when the current level/priority drops sufficiently low. This is why splraise() is much simpler than spllower().

This level/priority variable appears to be named differently in different places and versions of the code: CPL (not to be confused with CPU's current privilege level), SPL(?), ILEVEL.

This is my current, limited understanding of the implementation. There are more details to it.

Here are some clues that I've found and used for the answer:

Alexey Frunze
  • 61,140
  • 12
  • 83
  • 180
  • SPL is short for [S]ystem interrupt [P]riority [L]evel – Michael Petch Feb 06 '19 at 18:59
  • @Alexey Frunze: All due respect, I already understand the aspects about blocking interrupts at a specific level. I want some pointers on the assembly code above. @ Peter Cordes mentioned of thread local storage when we see %fs register. So I would want to understand, how would that be linked to setting an IPL (interrupt priority level). I know, we need to make certain assumptions here, as to what is stored in that location pointed to by %fs:214. Could it be data structures with info on "current" processor? But how that would translate into real action? – ultimate cause Feb 07 '19 at 07:04
  • @ultimatecause Yes, `FS:0` points to the structure specific to the CPU (not some thread). It's common practice, even in the Windows kernel. – Alexey Frunze Feb 07 '19 at 08:59
  • @ultimatecause As for the _real action_, you have two kinds of events upon which things are done: calling `spl*()` functions (most notably, `spllower()`, which ensures servicing of already pending hardware interrupts) and hardware interrupts triggering execution of their ISRs (which mark low level/priority interrupts as pending and service high level/priority ones either immediately or in multiple steps (e.g. a part is done immediately in the ISR while another part can be queued to be done outside of the ISRs, in some thread)). – Alexey Frunze Feb 07 '19 at 10:29