Based on info in https://man7.org/linux/man-pages/man2/brk.2.html, the brk Linux system call works like this:
void *sys_brk(void *desired_break) { return (void*)syscall(SYS_brk, desired_break); }It changes the virtual memory mapping of the current process, by growing or shrinking the read-write data region. The break is the end of the data region. To get the beginning of the data region (beyond sections containing global variables such as
.dataand.bss), callsys_brk(NULL)near the beginning of the program.It always returns the new break (both success and failure), which may be different from desired_break, especially on failure, because then the break is unchanged.
Quoting a comment in musl-1.1.16/src/malloc/expand_heap.c :
defend against buggy brk implementations that can cross the stack.
My question: Does the brk system call in Linux ever create the data region which can overlap with the stack of the main thread?
More specifically, was there an overlap in early Linux versions? If there was, when was it fixed?
I'm not interested in kernel bugs which happen only with address randomization. I'm not interested in kernel bugs which happen only when a dynamic loader (e.g. /lib/ld-linux.so.2) is used.
I'm asking this because I'm writing a minimalistic malloc implementation, which uses sys_brk under the hood, for Linux only, and I'd like to keep it as short as possible. (Please note that in this question I'm not interested in malloc implementations not using sys_brk.) If in Linux there is no overlap between the data region and the stack, then I don't have to check for that in my malloc implementation, thus it becomes shorter.
