qemu invalid tss type

Question

I am switching to user mode from kernel mode in 32-bit protected mode (only segmentation not paging). When I make the switch and call an interrupt in userspace I get this error from QEMU:

This may indicate that pixbuf loaders or the mime database could not be found.
qemu: fatal: invalid tss type
EAX=00003fec EBX=00001000 ECX=00000002 EDX=00001810
ESI=00000000 EDI=00000000 EBP=00003ff8 ESP=00003fec
EIP=00000000 EFL=00000006 [-----P-] CPL=3 II=0 A20=1 SMM=0 HLT=0
ES =0023 00006000 7fffffff 00c7f300 DPL=3 DS   [-WA]
CS =001b 00006000 7fffffff 00c7fa00 DPL=3 CS32 [-R-]
SS =0023 00006000 7fffffff 00c7f300 DPL=3 DS   [-WA]
DS =0023 00006000 7fffffff 00c7f300 DPL=3 DS   [-WA]
FS =0023 00006000 7fffffff 00c7f300 DPL=3 DS   [-WA]
GS =0023 00006000 7fffffff 00c7f300 DPL=3 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     00007c47 00000027
IDT=     00003000 0000ffff
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000
DR6=ffff0ff0 DR7=00000400
CCS=00000004 CCD=00003fe8 CCO=EFLAGS
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=0000000000000000 0000000000000000 XMM01=0000000000000000 0000000000000000
XMM02=0000000000000000 0000000000000000 XMM03=0000000000000000 0000000000000000
XMM04=0000000000000000 0000000000000000 XMM05=0000000000000000 0000000000000000
XMM06=0000000000000000 0000000000000000 XMM07=0000000000000000 0000000000000000

Here is the code that switches to user space:

switchSegments:
    
    mov ax, 0x23
    mov ds, ax
    mov es, ax
    mov fs, ax
    mov gs, ax

    mov eax, esp
    push dword 0x23
    push dword eax
    pushf
            
    push dword 0x1B
    push dword 0x0
    iret

Here is the GDT:

gdt_start:
    ;first entry is always null (each entry is 8 bytes or 64 bits)
    dd 0x0 ; 4 byte
    dd 0x0 ; 4 byte

; GDT for code segment. base = 0x00000000, length = 0xfffff
; for flags, refer to os-dev.pdf document, page 36
gdt_code: 
    dw 0xFFFF   ; segment length, bits 0-15
    dw 0x0       ; segment base, bits 0-15
    db 0x0       ; seg2ment base, bits 16-23
    db 10011010b ; flags (8 bits)
    db 11000000b ; flags (4 bits) + segment length, bits 16-19
    db 0x0       ; segment base, bits 24-31

; GDT for data segment. base and length identical to code segment
; some flags changed, again, refer to os-dev.pdf
gdt_data:
    dw 0xFFFF
    dw 0x0
    db 0x0
    db 10010010b
    db 11000000b
    db 0x0

U_code: 
    dw 0xFFFF    ; segment length, bits 0-15
    dw 0x6000    ; segment base, bits 0-15
    db 0x00      ; seg2ment base, bits 16-23
    db 11111010b ; flags (8 bits)
    db 11000111b ; flags (4 bits) + segment length, bits 16-19
    db 0x00       ; segment base, bits 24-31

; GDT for data segment. base and length identical to code segment
; some flags changed, again, refer to os-dev.pdf
U_data:
    dw 0xFFFF
    dw 0x6000
    db 0x00
    db 11110010b
    db 11000111b
    db 0x00


tssSegment:
    dw tssend - tss
    dw tss
    db 0x0
    db 0x89
    db 0x40
    db 0x0 

gdt_end:

; GDT descriptor
gdt_descriptor:
    dw gdt_end - gdt_start - 1 ; size (16 bit), always one less of its true size
    dd gdt_start ; address (32 bit)

I am calling the interrupt with int 2 from user space.

I'm not quite sure what invalid TSS means. I am not using paging for memory protection. Why am I getting this error and how can I fix it?

UPDATE 1

How does this TSS looks?

tss:
    dd 0x0
    
    dd 0x4000 ;esp0
    dd 0x10 ;ss0
    
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0
    dd 0x0

    dd 104 ;IOPB

    dd 0x0
tssend:

In order to switch from a less privileged (Like ring 3) to a more privileged ring (like ring 0) which happens if you are in user mode and you get/raise an exception or interrupts (including external interrupts) - you need to add a TSS entry in your GDT that points to a TSS structure. Without it CPU doesn't know what stack to use when transitioning. Once you have a TSS entry in your GDT you would use `LTR` instruction to load it into `TR` register. This all has to be set up in Ring 0 before you enter ring 3. Some information on setting one up is here: https://wiki.osdev.org/Task_State_Segment — Michael Petch, Apr 11 '23 at 02:12
You are getting the invalid TSS because you haven't loaded the `TR` (task register) with a selector in your GDT that is a TSS entry. — Michael Petch, Apr 11 '23 at 02:13
If you are using just ring3 and ring0 then the only transition that will occur is from Ring3 to Ring0 so the only fields you need to fill out in the TSS structure you create are SS0:ESP0. If you intend to allow Ring 3 to have direct access to port(s) then you would need an IOPB in your TSS structure as well, but this isn't needed to switch between rings (something to consider later on). If not using an IOPB put the size of your TSS structure in the `IOPB offset` field (the size of a TSS structure without an IOPB will be a value of 104) — Michael Petch, Apr 11 '23 at 02:29
@MichaelPetch I updated the question with a TSS struct. How does that look to you? — Jacob Knuth, Apr 11 '23 at 16:26
I also updated the GDT entry. When the interrupt is called, no error happens but nothing is put on the screen. I have the interrupts write to the screen, so i am not sure if they are being triggered. — Jacob Knuth, Apr 11 '23 at 17:22
There aren't enough `dd 0x00` before the IOBP which is in the wrong place. The last DD isn't needed if you aren't using CET. Here is an example TSS structure and example TSS entry: https://pastebin.com/4H6J5qiQ . You also have to remember to do the `ltr` instruction. Since it sounds like you aren't getting a TSS error now there will likely be something else wrong, but without seeing your code it may be hard to diagnose as to why you aren't seeing anything. Your last question might be better asked as a new question on SO. — Michael Petch, Apr 11 '23 at 17:39
I recommend using BOCHS for early testing like this because you can dump the TR/GDT/TSS, registers, and step through the instructions much easier to diagnose where the code went and why it may not have displayed anything. — Michael Petch, Apr 11 '23 at 17:44
thank you, I will look in to BOCHS. So it looks like the interrupt is being triggered. I added a line that I knew would cause the CPU to panic in the int handler. The CPU does panic when the user code invokes the interrupt and does not panic when the user codes does not invoke the interrupt. I write to video memory to put stuff on the screen address: 0xb8000. So for some reason it must be writing to a different spot in memory when coming from ring 3 to ring 0. I will create a new question. — Jacob Knuth, Apr 11 '23 at 17:55
https://stackoverflow.com/questions/75988850/video-memory-not-being-written-to-while-handling-interrupt-that-was-triggered-in — Jacob Knuth, Apr 11 '23 at 18:04
@MichaelPetch it is almost like the data segment is not being switched to kernel space. — Jacob Knuth, Apr 11 '23 at 18:16
Once the processor switches to kernel space you will have to set DS yourself to the appropriate value (whatever you kernel mode data selector is) in your interrupt/exception handlers. If you don't change DS yourself the value in DS will be whatever it was at the time of the interrupt (which would likely be your user mode DS). Before changing DS (or any segment registers) in the interrupt handler push the old value on the stack first and then restore them yo their original values before IRET'ing back. — Michael Petch, Apr 11 '23 at 18:24
yes I am using both c and assembly. let me take a look. Switching it manually is raising a protection fault error. I am not pushingthe old ds value yet. one sec — Jacob Knuth, Apr 11 '23 at 18:33

qemu invalid tss type

0 Answers0