3

I'm trying to create a Linux i386 a.out executable shorter than 4097 bytes, but all my efforts have failed so far.

I'm compiling it with:

$ nasm -O0 -f bin -o prog prog.nasm && chmod +x prog

I'm testing it in a Ubuntu 10.04 i386 VM running Linux 2.6.32 with:

$ sudo modprobe binfmt_aout
$ sudo sysctl vm.mmap_min_addr=4096
$ ./prog; echo $?
Hello, World!
0

This is the source code of the 4097-byte executable which works:

; prog.nasm
        bits 32
        cpu 386
        org 0x1000  ; Linux i386 a.out QMAGIC file format has this.

SECTION_text:
a_out_header:
        dw 0xcc  ; magic=QMAGIC; Demand-paged executable with the header in the text. The first page (0x1000 bytes) is unmapped to help trap NULL pointer references.
        db 0x64  ; type=M_386
        db 0  ; flags=0
        dd SECTION_data - SECTION_text  ; a_text=0x1000 (byte size of .text; mapped as r-x)
        dd SECTION_end - SECTION_data  ; a_data=0x1000 (byte size of .data; mapped as rwx, not just rw-)
        dd 0  ; a_bss=0 (byte size of .bss)
        dd 0  ; a_syms=0 (byte size of symbol table data)
        dd _start  ; a_entry=0x1020 (in-memory address of _start == file offset of _start + 0x1000)
        dd 0  ; a_trsize=0 (byte size of relocation info or .text)
        dd 0  ; a_drsize=0 (byte size of relocation info or .data)

_start:     mov eax, 4              ; __NR_write
        mov ebx, 1              ; argument: STDOUT_FILENO
        mov ecx, msg            ; argument: address of string to output
        mov edx, msg_end - msg  ; argument: number of bytes
        int 0x80                ; syscall

        mov eax, 1              ; __NR_exit
        xor ebx, ebx            ; argument: EXIT_SUCCESS == 0.
        int 0x80                ; syscall

msg:        db 'Hello, World!', 10
msg_end:

        times ($$ - $) & 0xfff db 0  ; padding to multiple of 0x1000  ; !! is this needed?
SECTION_data:   db 0
;       times ($$ - $) & 0xfff db 0  ; padding to multiple of 0x1000  ; !! is this needed?
SECTION_end:

How can I make the executable file smaller? (Clarification: I still want a Linux i386 a.out executable. I know that that it's possible to create a smaller Linux i386 ELF executable.) There is several thousands bytes of padding at the end of the file, which seems to be required.

So far I've discovered the following rules:

Thus file_size >= a_text + a_data >= 0x1000 + 1 == 4097 bytes.

The combinations nasm -f aout + ld -s -m i386linux and nasm -f elf + ld -s -m i386linux and as -32 + ld -s -m i386linux produce an executable of 4100 bytes, which doesn't even work (because its a_data is 0), and by adding a single byte to section .data makes the executable file 8196 bytes long, and it will work. Thus this path doesn't lead to less than 4097 bytes.

Did I miss something?

pts
  • 80,836
  • 20
  • 110
  • 183
  • 1
    Have you visited [Muppet Labs - A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux](https://www.muppetlabs.com/~breadbox/software/tiny/teensy.html)? You can write a working ELF file in 45 bytes. – David C. Rankin Oct 11 '22 at 21:50
  • @DavidC.Rankin: Indeed, as you show, it's possible to create a working Linux i386 hello-world executable which is smaller than 4097 bytes. However, the executable you suggest (hello_trysmall32) is not an answer to the question, because it's of the ELF format (`ld -m elf_i386`), but the question asks for a.out format. – pts Oct 11 '22 at 22:19
  • Ahh.. You are talking about [7.10 aout: Linux a.out Object Files](https://www.nasm.us/xdoc/2.09.10/html/nasmdoc7.html) – David C. Rankin Oct 11 '22 at 22:50
  • @DavidC.Rankin: The question asks about Linux a.out executable files (e.g. output of `ld -m i386linux`, see also direct output with `nasm -f bin` in the question). I don't care about the size of object files. – pts Oct 11 '22 at 23:03
  • 1
    I got you. It was clear after your first comment. I haven't jumped down the `a.out` rabbit hole far enough to help further. The nasm manual is a bit light on details, so this will likely end up being a Peter question `:)` – David C. Rankin Oct 12 '22 at 02:16

1 Answers1

2

TL;DR It doesn't work.

It is impossible to make a Linux i386 a.out QMAGIC executable shorter than 4097 bytes work on Linux 2.6.32, based on evidence in the Linux kernel source code of the binfmt_aout module.

Details:

  • If a_text is 0, Linux doesn't run the program. (Evidence for this check: a_text is passed as the length argument to mmap(2) here.)

  • If a_data is 0, Linux doesn't run the program. (Evidence for this check: a_data is passed as the length argument to mmap(2) here.)

  • If a_text is not a multiple of 0x1000 (4096), Linux doesn't run the program. (Evidence for this check: fd_offset + ex.a_text is passed as the offset argument to mmap(2) here. For QMAGIC, fd_offset is 0.)

  • If the file is shorter than a_text + a_data bytes, Linux doesn't run the program. (Evidence for this check: file sizes is compared to a_text + a_data + a_syms + ... here.)

  • Thus file_size >= a_text + a_data >= 0x1000 + 1 == 4097 bytes.

I've also tried OMAGIC, ZMAGIC and NMAGIC, but none of them worked. Details:

  • For OMAGIC, read(2) is used instead of mmap(2) within here, thus it can work. However, Linux tries to load the code to virtual memory address 0 (N_TXTADDR is 0), and this causes SIGKILL (if non-root and vm.mmap_min_addr is larger than 0) or SIGILL (otherwise), thus it doesn't work. Maybe the reason for SIGILL is that the page allocated by set_brk is not executable (but that should be indicated by SIGSEGV), this could be investigated further.

  • For ZMAGIC and NMAGIC, read(2) instead of mmap(2) within here if fd_offset is not a multiple of the page size (0x1000). fd_offset is 32 for NMAGIC, and 1024 for ZMAGIC, so good. However, it doesn't work for the same reason (load to virtual memory address 0).

  • I wonder if it's possible to run OMAGIC, ZMAGIC or NMAGIC executables at all on Linux 2.6.32 or later.

pts
  • 80,836
  • 20
  • 110
  • 183