Musl mmap ENOMEM

Question

I compiled some cross compilers against musl(x86_64, i686, arm). I need to compile code, that allocate like 2048 Mb +- 200Mb. However I noticed some errors with i686 musl compiler:

#include <stdio.h>
#include <sys/mman.h>
#include <sys/resource.h>
#include <errno.h>
int main(){
  void*ptr = mmap(0, 2147483647,PROT_READ|PROT_WRITE, 
            MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);       // malloc(2147483647)
  if(ptr == MAP_FAILED){
    printf("ERROR MMAP - "); 
    if(errno == ENOMEM){
      struct rlimit ra, rd; 
      if( getrlimit(RLIMIT_AS, &ra) < 0)
        printf("RLIMIT AS ERR\n");  
      if( getrlimit(RLIMIT_DATA, &rd) < 0)
        printf("RLIMIT AS ERR\n");   
      printf("ENOMEM, RLIMIT_AS: (%lu:%lu); RLIMIT_DATA: (%lu:%lu)\n", ra.rlim_cur, ra.rlim_max, rd.rlim_cur, rd.rlim_max); 
    }
  }
  else 
    printf("SUCCESS\n"); 
}

I know that there is no mistake with building i686-linux-musl. To prove it I, for example, downloaded i686-linux-musl from https://musl.cc/

I dont know why ldd tell me, that c is static, but file tell me it's dynamically linked.

However if I compile it with i686-linux-gnu (i586)

What's the problem with musl ? EDITED. I used strace and got. i686-linux-musl:

execve("./c", ["./c"], 0x7ffe6a24f548 /* 61 vars */) = 0
strace: [ Process PID=1723318 runs in 32 bit mode. ]
set_thread_area({entry_number=-1, base_addr=0xf7ff75ac, limit=0x0fffff, seg_32bit=1, contents=0, read_exec_only=0, limit_in_pages=1, seg_not_present=0, useable=1}) = 0 (entry_number=12)
set_tid_address(0xf7ff7654)             = 1723318
prlimit64(0, RLIMIT_AS, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_DATA, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
ioctl(1, TIOCGWINSZ, {ws_row=40, ws_col=235, ws_xpixel=0, ws_ypixel=0}) = 0
writev(1, [{iov_base="ERROR MMAP - ENOMEM, RLIMIT_AS: "..., iov_len=92}, {iov_base=")\n", iov_len=2}], 2ERROR MMAP - ENOMEM, RLIMIT_AS: (4294967295:4294967295); RLIMIT_DATA: (4294967295:4294967295)
) = 94
exit_group(0)

Whereis mmap?? And with i586-linux-gnu:

execve("./c2", ["./c2"], 0x7fff3e161878 /* 61 vars */) = 0
strace: [ Process PID=1723806 runs in 32 bit mode. ]
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
mmap(NULL, 2147483647, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77f34000
write(1, "SUCCESS\n", 8SUCCESS
)                = 8
exit(8)                                 = ?
+++ exited with 8 +++

Run both programs under `strace` and attach the relevant output. The two compilers are most likely placing your program at different virtual addresses. In the first case you have 2GiB of contiguous free space, in the second case you don't, because the compiler chose a virtual address in the middle of the virtual memory space. — Marco Bonelli, Jan 12 '23 at 13:24

score 4 · Accepted Answer · answered Jan 12 '23 at 13:44

4

By looking at the source code of musl's mmap we see that it forces the size to be less than PTRDIFF_MAX, which is probably 2147483647, and returns the ENOMEM error if it's not.

Ask for 1 byte less, and it will actually try to mmap.

However, the memory fragmentation concern still applies. On a 32-bit architecture, allocating 2GB of memory may succeed, but it won't always succeed and you have to be prepared for the possibility that it may not. Many 32-bit operating systems limit you to a maximum of 2GB of address space - not 4GB - including the memory you allocate and your program itself. On those operating systems, you won't ever be able to allocate 2GB. I think 32-bit Windows and 32-bit Linux work this way.

answered Jan 12 '23 at 13:44

user253751

57,427
7
48
90

Good catch. Besides, [the message in the commit that introduced this check in musl](https://github.com/bminor/musl/commit/3cd6f5229f079f892411e82fce3fe15c78eef4d8) gives a good explanation as to why one should avoid such large contiguous allocations. – Marco Bonelli Jan 12 '23 at 13:56
Yes, it's true. The PTRDIFF_MAX for i386, arm and etc is INT32_MAX, I cant mmap >= 2147483647. – Rainbow Jan 12 '23 at 13:57
But, the most wtf is: if there is free(), I can malloc maximum 2147479550. However if there is no free(), I can malloc() 2147483647. That's a jungle of musl ) – Rainbow Jan 12 '23 at 14:01
@Rainbow not sure what you mean – user253751 Jan 12 '23 at 14:02
@MarcoBonelli the check is faulty, since the mmap syscall rounds up to a page size. When you ask for 2147483646 bytes you get 214748364**8** bytes which is above the limit :) – user253751 Jan 12 '23 at 14:03
1

BTW: The hint here was that mmap doesn't appear in strace. Thanks to @MarcoBonelli for asking you to run strace. – user253751 Jan 12 '23 at 14:03
@user253751. `void*ptr = malloc(2147483646);` - work. However: `{ void*ptr = malloc(2147483646); free(ptr); }` - doesnt work – Rainbow Jan 12 '23 at 14:05
@user253751, yep, malloc(XXX) rount up to 2147483648 if XXX > 2147479550 – Rainbow Jan 12 '23 at 14:07
@user253751 LOL you are most definitely right. Someone should tell them :') – Marco Bonelli Jan 12 '23 at 14:08

score 2 · Answer 2 · answered Jan 12 '23 at 13:24

What's happening there is that the address space is fragmented. While you may still have enough free memory and enough free address space for the process, there is simply not a 2GB long contiguous range of unused addresses. Therefore, the mmap call fails, as it should. The different libraries of course have different allocation patterns and will get loaded at different addresses (or even random addresses with ASLR), so the size of the largest free block of address space will vary (it can even be random on purpose with ASLR).

You have to allocate the memory in smaller non-contiguous chunks, whether you like it or not. Alternatively, if you compile for 64 bit, you get a much larger address space where it's no problem at all to fit in a 2GB allocation.

But it's all 32bit compilers, is it bcs of different virtual addresses? Can I connect these chunks in one block memory, like i malloced it? — Rainbow, Jan 12 '23 at 13:41
No, you can't connect them. They're at physically different addresses. Look at `/proc//maps` for your process to check out what's been mapped where. — Jonathan S., Jan 12 '23 at 13:41

Musl mmap ENOMEM

2 Answers2