Small model DOS .exe compiled and linked by OpenWatcom crashes

Question

I'm trying to create a small DOS .exe program. I wrote the entry point in NASM assembly

; st.nasm
global _small_code_
global _printmsg_
extern _main0_

segment code
_small_code_:
..start:
mov ax, data
mov ds, ax
mov ax, stack
mov ss, ax
mov sp, stacktop

mov ah, 9  ; WRITE_STDOUT
mov dx, hello_msg
int 0x21
call _main0_
; call _printmsg_
; mov ax, 3
mov dx, ax
add dx, hello_msg
mov ah, 9  ; WRITE_STDOUT
int 0x21
mov ah, 0x4c  ; EXIT, exit code in al
int 0x21

_printmsg_:
ret
push dx
xchg ax, dx
mov ah, 9  ; WRITE_STDOUT
mov dx, hello_msg  ; !!
int 0x21
pop dx
ret  ; !! restore AX?

segment data
hello_msg: db 'Hello, World!', 13, 10, '$'

segment stack stack
resb 1024
stacktop:

Please note that I'm not sure how the ..start: code and the segments should look like, I copy-pasted that part from somewhere.

I wrote the main program in C:

/* prog.c */
void _printmsg(const char *msg);
int add(int a, int b) {
  return a + b * 2;
}
void other() {
  _printmsg("Hello!\r\n$"); /*CRASH*/
  /*_printmsg(0);*/  /*OK*/
}
int _main0() {
  return 5;
}

I compile it with this:

$ nasm -f obj -o st.obj st.nasm
$ owcc -bdos -mcmodel=s -fno-stack-check -Os -s -march=i86 -o prog.exe prog.c st.obj

The resulting prog.exe is:

$ xxd prog.exe 
00000000: 4d5a 8b00 0100 0200 0300 4000 ffff 0100  MZ........@.....
00000010: 4b04 0000 0a00 0100 2000 0000 0000 0000  K....... .......
00000020: 0b00 0100 1000 0100 0000 0000 0000 0000  ................
00000030: d1e2 01d0 c3b8 0000 e934 00b8 0500 c300  .........4......
00000040: 4865 6c6c 6f21 0d0a 2400 b801 008e d8b8  Hello!..$.......
00000050: 0100 8ed0 bc4b 04b4 09ba 3b00 cd21 e8da  .....K....;..!..
00000060: ff89 c281 c23b 00b4 09cd 21b4 4ccd 21c3  .....;....!.L.!.
00000070: 5292 b409 ba3b 00cd 215a c348 656c 6c6f  R....;..!Z.Hello
00000080: 2c20 576f 726c 6421 0d0a 24              , World!..$

Disassembly of prog.exe:

$ ndisasm -e 0x20 -b 16 prog.exe 
00000000  0B00              or ax,[bx+si]
00000002  0100              add [bx+si],ax
00000004  1000              adc [bx+si],al
00000006  0100              add [bx+si],ax
00000008  0000              add [bx+si],al
0000000A  0000              add [bx+si],al
0000000C  0000              add [bx+si],al
0000000E  0000              add [bx+si],al
00000010  D1E2              shl dx,1
00000012  01D0              add ax,dx
00000014  C3                ret
00000015  B80000            mov ax,0x0
00000018  E93400            jmp 0x4f
0000001B  B80500            mov ax,0x5
0000001E  C3                ret
0000001F  004865            add [bx+si+0x65],cl
00000022  6C                insb
00000023  6C                insb
00000024  6F                outsw
00000025  210D              and [di],cx
00000027  0A24              or ah,[si]
00000029  00B80100          add [bx+si+0x1],bh
0000002D  8ED8              mov ds,ax
0000002F  B80100            mov ax,0x1
00000032  8ED0              mov ss,ax
00000034  BC4B04            mov sp,0x44b
00000037  B409              mov ah,0x9
00000039  BA3B00            mov dx,0x3b
0000003C  CD21              int 0x21
0000003E  E8DAFF            call 0x1b
00000041  89C2              mov dx,ax
00000043  81C23B00          add dx,0x3b
00000047  B409              mov ah,0x9
00000049  CD21              int 0x21
0000004B  B44C              mov ah,0x4c
0000004D  CD21              int 0x21
0000004F  C3                ret
00000050  52                push dx
00000051  92                xchg ax,dx
00000052  B409              mov ah,0x9
00000054  BA3B00            mov dx,0x3b
00000057  CD21              int 0x21
00000059  5A                pop dx
0000005A  C3                ret
0000005B  48                dec ax
0000005C  656C              gs insb
0000005E  6C                insb
0000005F  6F                outsw
00000060  2C20              sub al,0x20
00000062  57                push di
00000063  6F                outsw
00000064  726C              jc 0xd2
00000066  64210D            and [fs:di],cx
00000069  0A24              or ah,[si]

prog.exe puts DOSBox to an infinite loop. Oddly enough, if I remove the string literal from the C source file (in the other function, which isn't even called), it successfully returns. What's wrong in the assembly file?

Please note that this is the first time I'm using OpenWatcom, and this is the first time I build a DOS .exe file.

I don't want to write a main function, because that would cause the OpenWatcom libc to be linked to the output executable, making it unnecessarily large.

Michael Petch · Accepted Answer · 2020-07-02T17:13:10.717

The primary problem is in how you define the code segment. The Watcom C/C++ compiler when using the SMALL memory model requires the code segment to be called _TEXT with a class of CODE. This mismatch between the assembly code and the C code leads to the code segment being in different physical segments and the call _main0_ jumping to the wrong place in memory causing exceptions to be thrown and the program hanging or crashing.

You can also get the Watcom linker to generate the required STACK in the DOS EXE by creating a segment called _STACK with attribute STACK and class STACK. If you create the stack segment this way, you won't need to initialize SS:SP at the beginning of your program.

The other sections that Watcom uses in the SMALL memory model are:

_DATA segment with a class of DATA for read/write data
CONST segment with a class of DATA for string literals (that aren't expected to be modified)
CONST2 segment with a class of DATA for other read only data
_BSS segment with class of BSS for uninitialized data.

Watcom expects that the segments CONST, CONST2, _DATA and _BSS to all be in the same group called DGROUP. All the data in the same group can be referenced by the name of the group. When you set up DGROUP in the way Watcom expects then all you have to do is initialize DS to the DGROUP segment and not the individual segments within the group.

The special label that starts with .. acts as the DOS entry point where execution should start. Thus ..start is used to generate an entry point in the DOS EXE header telling the program loader where to start executing when the program is loaded into memory.

A revised version of your assembly code could have looked like this:

; st.nasm

; DGROUP in watcom C/C++ for small model is:
GROUP DGROUP CONST CONST2 _DATA _BSS

global _small_code_
global _printmsg_
extern _main0_

; Code Segment (16-bit code)
segment _TEXT use16 class=CODE
_small_code_:
; .. denotes the label to be used as the DOS entry point
..start:
    mov ax, DGROUP
    mov ds, ax

    mov ah, 9  ; WRITE_STDOUT
    mov dx, hello_msg
    int 0x21
    call _main0_
    ; call _printmsg_
    ; mov ax, 3
    mov dx, ax
    add dx, hello_msg
    mov ah, 9  ; WRITE_STDOUT
    int 0x21
    mov ah, 0x4c  ; EXIT, exit code in al
    int 0x21

_printmsg_:
    ret
    push dx
    xchg ax, dx
    mov ah, 9  ; WRITE_STDOUT
    mov dx, hello_msg  ; !!
    int 0x21
    pop dx
    ret  ; !! restore AX?

; Read only string literals here
segment CONST class=DATA
hello_msg: db 'Hello, World!', 13, 10, '$'

; Other read only data here
segment CONST2 class=DATA

; Read/Write data here
segment _DATA class=DATA

; Uninitialized data segment
segment _BSS class=BSS

; Stack segment 1k in size
segment _STACK STACK class=STACK
resb 1024

This code assumes that SS != DS, however it would have to be compiled with OWCC's option -Wc,-zu that passes the -zu to WCC (Watcom Compiler). -zu modifies code generation so that:

-zu             SS != DGROUP (i.e., do not assume stack is in data segment)

If you wish to set SS==DS==DGROUP there are a number of ways to do it. One option I may suggest is putting _STACK in DGROUP with all the other program data. You would need a label after resb 1024 like stack_top: so you can load that offset into SP at startup after you set SS to the same value as DS. This change would result in assembly code that looks like:

; st.nasm

; DGROUP in watcom C/C++ for small model is:
GROUP DGROUP CONST CONST2 _DATA _BSS _STACK
; _STACK has been added to DGROUP so we can set SS==DS==DGROUP

global _small_code_
global _printmsg_
extern _main0_

; Code Segment (16-bit code)
segment _TEXT use16 class=CODE
_small_code_:
; .. denotes the label to be used as the DOS entry point
..start:
    mov ax, DGROUP
    mov ds, ax
    mov ss, ax             ; Set stack SS:SP to DGROUP:stack_top
    mov sp, stack_top

    mov ah, 9  ; WRITE_STDOUT
    mov dx, hello_msg
    int 0x21
    call _main0_
    ; call _printmsg_
    ; mov ax, 3
    mov dx, ax
    add dx, hello_msg
    mov ah, 9  ; WRITE_STDOUT
    int 0x21
    mov ah, 0x4c  ; EXIT, exit code in al
    int 0x21

_printmsg_:
    ret
    push dx
    xchg ax, dx
    mov ah, 9  ; WRITE_STDOUT
    mov dx, hello_msg  ; !!
    int 0x21
    pop dx
    ret  ; !! restore AX?

; Read/Write data here
segment _DATA class=DATA

; Read only string literals here
segment CONST class=DATA
hello_msg: db 'Hello, World!', 13, 10, '$'

; Other read only data here
segment CONST2 class=DATA

; Uninitialized data segment
segment _BSS class=BSS

; Stack segment 1k in size
segment _STACK STACK class=STACK
resb 1024
stack_top:

This worked like a charm. I had to adjust `ss` (and also `sp` accordingly) to make it equal to `ds`, without that pointers to local variables didn't work. — pts, Jun 20 '20 at 17:55
@pts: I've amended my answer with an addition about producing a version where SS==DS==DGROUP . I haven't tested it but it should work. — Michael Petch, Jun 20 '20 at 18:44
@pts : If you remove `mov ss, ax` and `mov sp, stack_top` and use the defaults then the compiler could put SS in a different segment than DS. If that happens the program will appear to work but the top of the stack actually wouldn't be at the right physical memory address. — Michael Petch, Jun 24 '20 at 15:55
To get better control over the .exe header fields, I ended up writing my own linker targeting DOS .exe (with the small memory model) and DOS .com (with the tiny memory model). Since my linker fills the ss and sp .exe header fields correctly, it's enough to do `push ss` and `pop ds` at the entry point. Because of this, the .exe file doesn't have any explicit relocations either. — pts, Jun 25 '20 at 00:55
@ecm: My linker is part of https://github.com/pts/dosmc , it's implemented in the link_executable function. — pts, Jun 21 '22 at 10:28

Small model DOS .exe compiled and linked by OpenWatcom crashes

1 Answers1