3

This is a follow-up from (nasm x86 real mode) How to write/read strings in boot-loaded sector?.

I'm working on a toy OS for x86 real mode in NASM. The 512B boot sector loads another sector with the rest of the code. The problem is that I seem to run out of space at the end of the program.

Here's the beginning of the source file:

;;; nasm -f bin -o boot.bin os.asm
;;; qemu-system-x86_64 boot.bin

    bits 16

    section boot, vstart=0x0000

    ;; Load next sector.
    ;; adapted from:
    ;; https://blog.benjojo.co.uk/post/interactive-x86-bootloader-tutorial
    mov ah, 0x02
    mov al, 1   
    mov ch, 0    
    mov cl, 2    
    mov dh, 0   
    mov bx, newsector 
    mov es, bx  
    xor bx, bx
    int 0x13
    jmp newsector:0

    newsector equ 0x0500

    times 510-($-$$) db 0
    db 0x55
    db 0xaa

    section os, vstart=0x0000
    mov ax, newsector
    mov ds, ax

And here's the end of the source file, where I store the data. There's a keymap to convert input according to the Dvorak keyboard layout. But the program seems to get "cut off" after fu.

    input times 16 db 0
    repl_prompt times 16 db 0

    dvorak db 1

dvorak_keymap:
    db "aa"
    db "nb"
    db "ic"
    db "hd"
    db "de"
    db "yf"
    db "ug"
    db "jh"
    db "gi"
    db "cj"
    db "vk"
    db "pl"
    db "mm"
    db "ln"
    db "so"
    db "rp"
    db "xq"
    db "or"
    db ";s"
    db "kt"
    db "fu"
    db ".v"     ; FIXME: gets cut off here
    db ",w"
    db "bx"
    db "ty"
    db "/z"
    db 0

So when you boot the OS, you can type keys a-u in Dvorak but not v-z. Furthermore, calling the program's print function on the dvorak_keymap string confirms that the string terminates after fu.

But more importantly, this indicates that I've run out of space in my program so I can't add any more data or code. I can't have hit the 1 MB memory limit because the source file is only 282 sloc.

I'm guessing it has something to do with how I load the sectors? I'm new to assembly and low-level programming so any help would be greatly appreciated.

source: https://github.com/jtherrmann/os/blob/master/os.asm

raw: https://raw.githubusercontent.com/jtherrmann/os/master/os.asm

Edit: Additionally, when I add more data/code higher up in the file, the keymap gets cut off earlier, and when I remove higher up data/code, the keymap gets cut off later or not at all. So I know that this has something to do with space limitations.

jth
  • 369
  • 3
  • 8
  • 3
    In the firststage you call int 0x13/ah=2. The register `al` is the number of sectors to read. The value 1 reads a single 512 byte sector. You will have to increase AL from 1 to a number of sectors that is large enough to encompass your kernel. As it is you only read 512 bytes (1 sector) and the rest is never loaded in memory making it appear it is cut off. – Michael Petch Sep 23 '18 at 06:53
  • 2
    *because the source file is only 282 sloc*. That's a useless way to count binary size unless you know what kind of source lines there are. `times 1024*1024 dd 0` assembles to 4MiB of zeros. Just look at the NASM listing from `nasm -l/dev/stdout foo.asm`, or look at the size of the resulting binary, to find out how many sectors you need to load. – Peter Cordes Sep 23 '18 at 06:57
  • 1
    @MichaelPetch That fixed it. I can't thank you enough. Both of your answers have saved me a lot of frustration. Thank you! – jth Sep 23 '18 at 07:02
  • @PeterCordes Hm, that's a very good point. Thank you for explaining that. – jth Sep 23 '18 at 07:03
  • 2
    You can get the assembler to compute the number of sectors needed to load by creating a label at the start of the kernel code and a label at the end of it. If you subtract the end from the beginning (giving you size in bytes if the kernel) you can then compute the number of 512 byte sectors. I've made that adjustment to your code here: http://www.capp-sysware.com/misc/stackoverflow/52461308/boot.asm – Michael Petch Sep 23 '18 at 07:04
  • @MichaelPetch Fantastic, thanks! – jth Sep 23 '18 at 07:08
  • 2
    The catch is that you are limited to a maximum of 63 as a value of _AL_. That will be good enough to get you up to a kernel size of 63*512=32,256. It's actually a little lower because 0x500+63*512 has you loading the kernel on top of the bootsector at 0x7c00 (it overlaps). So really your kernel can be up to about 59 sectors (not 63). There is a way around this but it requires considerable modification to your first stage to relocate itself and then load the kernel. – Michael Petch Sep 23 '18 at 07:11
  • 1
    I had amended the code at the link above to give an example of setting the stack to something that will be out of the way (for now). By the way, I was wrong above. Your kernel is being loaded at 0x500:0x0000 which is physical address 0x5000 which is considerably closer to 0x7c00 then what I suggested above. I had mistakenly assumed it was 0x0050. Using a `newstart` of 0x0060 would put your kernel as low down in memory to avoid the interrupt vector table and the BIOS Data Area of all the BIOSes that have been produced. – Michael Petch Sep 23 '18 at 07:43
  • 1
    You made it ("only 282 sloc") sound like 282 lines of code is not much... :) ... also in real mode you don't have 1MiB of RAM available for yourself. Some areas are reserved for video buffer, interrupt vector table, BIOS data, and similar. Also you may want to leave some RAM for user apps, actually most of it. MS-DOS did occupy under 100kiB of base RAM IIRC (actually could have been squeezed down to 20-30kiB with some parts offloaded into "high memory" I think.. can't recall the exact numbers any more). If you are planning to reach mega-byte sizes, then switch to 32b protected mode early. – Ped7g Sep 23 '18 at 08:18
  • @MichaelPetch Thanks for the clarification. I copied the value of `newstart` from another source so I had no idea what I was doing. – jth Sep 24 '18 at 00:37
  • 1
    @Ped7g Ah, thank you for the explanation. This is just a small school project so I'm not even bothering to separate the kernel from the rest of the "applications". I did a bit of research on setting up protected mode but decided it was beyond my current level of knowledge. Right now I just want an environment for implementing neat interactive commands. :) – jth Sep 24 '18 at 00:45

1 Answers1

5

Your code has multiple mistakes.

a) It doesn't set up a stack at a known location, and then loads data from disk at an address that might overwrite the stack (that might be at the same address).

b) It assumes that loading data from disk worked without checking if the BIOS returned an error.

c) It only loads one sector without any way to determine the actual size and "self-adjust"; so as the file grows it's going to be a continual maintenance chore. This is likely to be the reason why your data gets "cut off" (everything after the first 512 bytes are not loaded)

d) You use "newstart" before it's been defined, and then define "newstart equa 0x0500" after it's too late.

e) You assume all characters are printable. They aren't (cursor movement, function keys, ...)

Also note that your code to convert a character from QWERTY to Dvorak is incredibly inefficient (looping through all entries to find the right one instead of using the original character as an index into a table - e.g. like movzx eax,al, mov al,[table+eax]); and it currently won't work for a lot of cases (e.g. if the user presses "shift+a" or if they have capslock on).

Brendan
  • 35,656
  • 2
  • 39
  • 66
  • 2
    As for part D,NASM is a multipass assembler and can resolve the forward reference of the EQU. It is a bit unorthodox I agree but it should work. – Michael Petch Sep 23 '18 at 07:14
  • @MichaelPetch: It's a mistake (e.g. horrible for code maintenance) regardless of whether it's a bug or not. This is why I was too lazy to check if the pre-processor (and not the assembler) is able to deal with it. – Brendan Sep 23 '18 at 07:16
  • That is why my comment mentions it being unorthodox. – Michael Petch Sep 23 '18 at 07:16
  • @Brendan: `equ` is not part of NASM's preprocessor (`%define newsector 0x0500` would need to appear before use), it's part of the regular assembler which handles labels, and `label2-label1` assemble-time-constant expressions. But +1 especially for the comment about the Dvorak table. The a-z should definitely be implicit by position. – Peter Cordes Sep 23 '18 at 07:21
  • Thanks for pointing these out. I don't know much about assembly so the feedback is very useful. I'll particularly look into fixing a) because it seems the most critical. Thanks also for the example of improving the Dvorak table, I'll definitely look at incorporating that. – jth Sep 23 '18 at 07:29
  • As the index into the lookup table is only unsigned 8 bit, there exists specialized x86 instruction for that... [`XLATB` ladies and gentlemans](http://www.felixcloutier.com/x86/XLAT:XLATB.html) (OP: this is semi-joke, doing a bit of fun of x86 ISA having all kind of weird-today-unused instructions, in modern code this is obsolete and the modern CPUs usually implement it in slow way, so modern code often uses simple `mov` like Brendan shows - but in your particular case this can be actually very effective (saving you code size and lot more performant than your loop)). – Ped7g Sep 23 '18 at 08:25
  • 1
    Brendan and @MichaelPetch: I've updated my code to fix mistakes a, c, and d (not b or e), plus the improved Dvorak table lookup. Further feedback/criticism is welcome, and thanks again for your help! https://github.com/jtherrmann/os/blob/master/os.asm – jth Sep 24 '18 at 00:28