Embedded assembly in C (DOS) - illegal instruction

Question

I'm trying to re-program instruction vector table. Here is the code I use:

#include <stdio.h>

int a=1;
void func();

void keyboard()
{
    printf("\n\nkeyboard!!!\n");
    a=0;
    asm{iret}
}

int main ()
{
    printf("starting...");
    func();
    return 0;
}

      int vectorcs = 0;
 int vectorip = 0;

void func()
{

    printf("\n*****\n");
    asm{
        cli
        mov ax,0
        mov es,ax
        mov bx,36
        mov ax,word ptr es:[bx]
        mov vectorip,ax
        push ax
         mov ax,word ptr es:[bx+2]
        mov vectorcs,ax
        push ax
        mov ax,cs
        mov word ptr es:[bx],offset keyboard
        mov es:[bx+2],ax
        sti
    }
    printf("\n%d %d\n",vectorip,vectorcs);

    while (a) {
    }
    asm {
        cli
        mov es,bx
        mov bx,36
        pop ax
        mov word ptr es:[bx+2],ax
    }
    asm{
        pop ax
        mov word ptr es:[bx],ax
        sti
    }
}

I'm using Turbo C++ 3.0 When I try to run this program, "16 Bit MS-DOS Subsystem: The NTVDM CPU has encountered an illegal instruction." appears. Then it shows contents of CS, OP, and IP registers. I can't continue the program. Any suggestions?

Answer 1

What you're doing is not right for multiple reasons:

1. Regular C functions can't be safely used as interrupt service routines because they don't correctly save, load and restore the CPU registers. They must be declared with the interrupt keyword. And they'll have iret for you at the end.
2. Variables that can change in the program asynchronously from interrupt routines must be declared as volatile, otherwise you're risking to have accesses to them incorrectly optimized out by the compiler.
3. Your inline assembly code probably corrupts the contents of CPU registers. One thing that's wrong with this code is that your asm blocks mess with the stack pointer. The first block exits with several extra words on the stack. This may be completely unexpected for the compiler and can break your program. There may be other issues, but I'm not going to check with the compiler documentation which registers must be preserved by inline assembly blocks. I'd avoid doing this altogether and opt for the setvect() function instead.
4. Calling most of standard library functions from inside of interrupt service routines is asking for trouble because these functions generally aren't reentrant/thread-safe. They can modify some global variables or states in completely unexpected ways for the rest of the program. The same is true for calling DOS service functions from the interrupt service routines (which your printf() relies on, btw). You can only call those when DOS says it's OK. It does so via the InDos flag variable and still, not all are safe to call when InDos=0.

See how to change interrupt vectors, define interrupt service routines and call DOS functions from them, all with Turbo C, in the answer to this question.

You may also find this question and its answers useful.

EDIT:

This is how you do it without dos.h's functionality with inline asm:

#include <stdio.h>

volatile int a = 1;
void interrupt (*pOldInt9)(void);
void func(void);

void interrupt keyboard(void)
{
    printf("\n\nkeyboard!!!\n");

    asm {
        in  al, 0x60
        in  al, 0x61
        mov ah, al
        or  al, 0x80
        out 0x61, al
        mov al, ah
        out 0x61, al
    }

    a = 0;

    asm {
        mov al, 0x20
        out 0x20, al
    }
}

int main(void)
{
    printf("starting...");
    func();
    return 0;
}

void func(void)
{
    printf("\n*****\n");

    asm {
        push    bx
        push    es

        mov     bx, 9 * 4
        mov     ax, 0
        mov     es, ax

        cli

        mov     ax, es:[bx]
        mov     word ptr pOldInt9, ax
        mov     word ptr es:[bx], offset keyboard

        mov     ax, es:[bx + 2]
        mov     word ptr pOldInt9[2], ax
        mov     es:[bx + 2], cs

        sti

        pop     es
        pop     bx
    }

    while (a) {}

    asm {
        push    bx
        push    es

        mov     bx, 9 * 4
        mov     ax, 0
        mov     es, ax

        cli

        mov     ax, word ptr pOldInt9
        mov     es:[bx], ax

        mov     ax, word ptr pOldInt9[2]
        mov     es:[bx + 2], ax

        sti

        pop     es
        pop     bx
    }
}

Answer 2

asm {
    cli
    mov es,bx
    mov bx,36
    pop ax
    mov word ptr es:[bx+2],ax
}

What does bx contain before that code?

void keyboard()
{
    printf("\n\nkeyboard!!!\n");
    a=0;
    asm{iret}
}

This function has set up a stack frame that you haven't correctly destroyed.