Starting 32 bit protected mode

Question

I'm trying to start 32 bit protected mode in assembly but when I run it in dosbox I get error: "JMP illegal descryptor type 10" and when I run it in real dos, whole PC just restarts. Whole code:

    org 100h
    cli

    lgdt [GdtDesc]

    mov eax,cr0
    or ax,1
    mov cr0,eax

    jmp $+2
    jmp 0x8:Start32

GdtDesc:
    dw 0x18
    dd Gdt

Start32:
    mov ax,10h
    mov ds,ax
    mov es,ax
    mov fs,ax
    mov gs,ax
    mov ss,ax

    mov esp,8000h

    mov eax,cr0
    xor eax,1
    mov cr0,eax

    ret

ALIGN 4 ; Global Descriptor Table (GDT) (faster accessed if aligned on 4).
Gdt:   ; GDT[0]: Null entry, never used.
    dd 0
    dd 0
    ; GDT[1]: Executable, read-only code, base address of 0, limit of FFFFFh, 
    ; granularity bit (G) set (making the limit 4GB)

    dw 0FFFFh ; Limit[15..0]
    dw 0000h ; Base[15..0]
    db 00h ; Base[23..16]
    db 10011010b ; P(1) DPL(00) S(1) 1 C(0) R(1) A(0)
    db 11001111b ; G(1) D(1) 0 0 Limit[19..16]
    db 00h ; Base[31..24]
    ; GDT[2]: Writable data segment, covering the save address space than GDT[1].

    dw 0FFFFh ; Limit[15..0]
    dw 0000h ; Base[15..0]
    db 00h ; Base[23..16]
    db 10010010b ; P(1) DPL(00) S(1) 0 E(0) W(1) A(0)
    db 11001111b ; G(1) B(1) 0 0 Limit[19..16]
    db 00h ; Base[31..24]

@sivizius Well, there is nothing wrong with using 32 bit registers in your 16 bit code. What's the problem? — fuz, Jun 19 '18 at 18:22
`org 100h` makes your code look like your trying to create a `.COM` file. Using a `.COM` file to start _Protected Mode_ is new to me, so I don't know if it's possible or even useful. — zx485, Jun 19 '18 at 18:52
However, you have a `CLI` at the beginning, but I don't see any `STI`. This can be a cause of various errors. — zx485, Jun 19 '18 at 18:52
You have a number of potential problems. The biggest one is that in your `GdtDesc` structure the address of `gdt` isn't sufficient because `lgdt` requires a descriptor record where the `gdt` address is a linear address. You would have to compute the `gdt` at run time by getting the value of `CS` multiply by 16 (or shift left by 4) and then add that to the address of `gdt` stored at `GdtDesc+2`. You will also need to adjust your FAR JMP into protected mode by computing the JMP address in a similar fashion as above at run time. — Michael Petch, Jun 19 '18 at 20:05
Jumping into protected mode the memory you have set up is flat so you need to compute the linear address to jump to. Once in protected mode the jmp targets aren't real mode segmented address (segment:address). — Michael Petch, Jun 19 '18 at 20:07
Once in protected mode you won't be able to do ` ret` to return to dos. You either will have to stay in an infinite loop after entering protected mode or return to real mode and then do the `ret`. This might be an XY problem. What is it you really are trying to use this code for? — Michael Petch, Jun 19 '18 at 20:08
The other thing to watch out for is that DOS may be running in VM8086 mode because of a memory manager. In that case you won't be able to switch into protected mode by changing cr0. You could make sure that EMM386.SYS (or EMM386.COM) are not loaded when DOS is booted. EMM386 enters protected mode and virtualizes DOS in a VM8086 task. — Michael Petch, Jun 19 '18 at 20:59
Some time back on IRC I wrote some code that did something similar. I cleaned it up a bit and put it on my server: http://www.capp-sysware.com/misc/ircasm/enterpm.asm . Not sure if it is of any use to you but you might get some inspiration. — Michael Petch, Jun 20 '18 at 02:34

score 3 · Accepted Answer · edited Jun 21 '18 at 01:58

Best answer is from Michael Petch (nice and clean code + notes):

Some time back on IRC I wrote some code that did something similar. I cleaned it up a bit and put it on my server: capp-sysware.com/misc/ircasm/enterpm.asm . Not sure if it is of any use to you but you might get some inspiration. – Michael Petch

; Assemble with NASM as
;     nasm -f bin enterpm.asm -o enterpm.com

STACK32_TOP EQU 0x200000
CODE32_REL  EQU 0x110000
VIDEOMEM    EQU 0x0b8000

use16
; COM program CS=DS=SS
org 100h

    call check_pmode    ; Check if we are already in protected mode
                        ;    This may be the case if we are in a VM8086 task.
                        ;    EMM386 and other expanded memory manager often
                        ;    run DOS in a VM8086 task. DOS extenders will have
                        ;    the same effect

    jz not_prot_mode    ; If not in protected mode proceed to switch
    mov dx, in_pmode_str;    otherwise print an error and exit back to DOS
    mov ah, 0x9
    int 0x21            ; Print Error
    ret

not_prot_mode:
    call a20_on         ; Enable A20 gate (uses Fast method as proof of concept)
    cli

    ; Compute linear address of label gdt_start
    ; Using (segment << 4) + offset
    mov eax,cs          ; EAX = CS
    shl eax,4           ; EAX = (CS << 4)
    mov ebx,eax         ; Make a copy of (CS << 4)
    add [gdtr+2],eax    ; Add base linear address to gdt_start address
                        ;     in the gdtr
    lgdt [gdtr]         ; Load gdt

    ; Compute linear address of label code_32bit
    ; Using (segment << 4) + offset
    add ebx,code_32bit  ; EBX = (CS << 4) + code_32bit

    push dword 0x08     ; CS Selector
    push ebx            ; Linear offset of code_32bit
    mov bp, sp          ; m16:32 address on top of stack, point BP to it

    mov eax,cr0
    or eax,1
    mov cr0,eax         ; Set protected mode flag

    jmp dword far [bp]  ; Indirect m16:32 FAR jmp with
                        ;    m16:32 constructed at top of stack
                        ;    DWORD allows us to use a 32-bit offset in 16-bit code

; 16-bit functions that run in real mode

; Check if protected mode is enabled, effectively checkign if we are
; in in a VM8086 task. Set ZF to 0 if in protected mode

check_pmode:
    smsw ax             ; Get lower 16 bits of control register in AX
    test ax, 0x1        ; Test the PE bit (bit 0) and set ZF flag accordingly
    ret 

; Enable a20 (fast method). This may not work on all hardware
a20_on:
    cli
    in al, 0x92         ; Read System Control Port A
    test al, 0x02       ; Test current a20 value (bit 1)
    jnz .skipfa20       ; If already 1 skip a20 enable
    or al, 0x02         ; Set a20 bit (bit 1) to 1
    and al, 0xfe        ; Always write a zero to bit 0 to avoid
                        ;     a fast reset into real mode
    out 0x92, al        ; Enable a20
.skipfa20:
    sti
    ret

in_pmode_str: db "Processor already in protected mode - exiting",0x0a,0x0d,"$"

align 4
gdtr:
    dw gdt_end-gdt_start-1
    dd gdt_start

gdt_start:
    ; First entry is always the Null Descriptor
    dd 0
    dd 0

gdt_code:
    ; 4gb flat r/w/executable code descriptor
    dw 0xFFFF           ; limit low
    dw 0                ; base low
    db 0                ; base middle
    db 0b10011010       ; access
    db 0b11001111       ; granularity
    db 0                ; base high

gdt_data:
    ; 4gb flat r/w data descriptor
    dw 0xFFFF           ; limit low
    dw 0                ; base low
    db 0                ; base middle
    db 0b10010010       ; access
    db 0b11001111       ; granularity
    db 0                ; base high
gdt_end:

; Code that will run in 32-bit protected mode
; Align code to 4 byte boundary. code_32bit label is
; relative to the origin point 100h
align 4
code_32bit:
use32
; Set virtual memory address of pm code/data to CODE32_REL
; We will be relocating this section from low memory where DOS
; originally loaded it.
section protectedmode vstart=CODE32_REL, valign=4
start_32:
    cld                 ; Direction flag forward
    mov eax,0x10        ; 0x10 is flat selector for data
    mov ds,eax
    mov es,eax
    mov fs,eax
    mov gs,eax
    mov ss,eax
    mov esp,STACK32_TOP ; Should set ESP to a usable memory location
                        ; Stack will be grow down from this location

    mov edi,start_32    ; EDI = linear address where PM code will be copied
    mov esi,ebx         ; ESI = linear address of code_32bit
    mov ecx,PMSIZE_LONG ; ECX = number of DWORDs to copy
    rep movsd           ; Copy all code/data from code_32bit to CODE32_REL
    jmp 0x08:.relentry  ; Absolute jump to relocated code

.relentry:
    mov ah, 0x57        ; Attribute white on magenta

    ; Print a string to display
    mov esi,str         ; ESI = address of string to print
    mov edi,VIDEOMEM    ; EDI = base address of video memory
    call print_string_attr

    cli
endloop:
    hlt                 ; Halt CPU with infinite loop
    jmp endloop

print_string_attr:
    push ecx
    xor ecx,ecx         ; ECX = 0 current video offset
    jmp .loopentry
.printloop:
    mov [edi+ecx*2],ax  ; Copy attr and character to display
    inc ecx             ; Next word position
.loopentry:
    mov al,[esi+ecx]    ; Get next character to print
    test al,al
    jnz .printloop      ; If it's not NUL continue
.endprint:
    pop ecx
    ret

str: db "Protected Mode",0

PMSIZE_LONG equ ($-$$+3)>>2
                        ; Number of DWORDS that the protected mode
                        ;    code and data takes up (rounded up)

Starting 32 bit protected mode

1 Answers1