3

I have a scenario where I need to map non-linear parts of a file, linearly in a process space.

For example, if file is 10 pages, I may need to map first 3, skip 4, and last 3. The mapping should be linear, s.t. incremental access in process space allows to go to page 8 of file after page 3, as page 4,5,6 & 7 were not mapped.

I want to know if this is possible in Linux.

Thanks.

Jake
  • 16,329
  • 50
  • 126
  • 202
  • Is this not just a case of calling `mmap` multiple times? – Oliver Charlesworth Jan 15 '13 at 21:10
  • Won't calling mmap multiple times give a pointer result for each mmap call ? I have the scenario where entire mapping needs to have just one pointer result. Is it in-feasible ? – Jake Jan 15 '13 at 21:12
  • It's been a while since I've done anything with `mmap`, but isn't one of the arguments an optional address? – Oliver Charlesworth Jan 15 '13 at 21:18
  • 1
    Yes, the first argument. So you are suggesting call mmap multiple times, but from 2nd call on wards, use the result from previous call as first argument ? – Jake Jan 15 '13 at 21:19
  • @OliCharlesworth will there be an issue for linear access in process space..from http://linux.die.net/man/2/mmap ... the kernel takes first argument as a hint about where to place the mapping; on Linux, the mapping will be created at a nearby page boundary. – Jake Jan 15 '13 at 21:23
  • I'm not really suggesting anything ;) I just had a vague feeling that you could map arbitrary parts of a file to arbitrary locations in virtual address space; so you could just map each file "segment" to a neighbouring address (on page boundaries, of course). But now I think about it, it's not clear how you'd choose those addresses in the first place... – Oliver Charlesworth Jan 15 '13 at 21:25
  • @OliCharlesworth: choosing the adresses is easy. just call malloc with a large enough buffer plus 2 times the page size and "hint" mmap to the start of the region. second call to mmap point to start plus page size and so on. linux should take care of the rest, if you're mapping only multiples of the page size, of course – sl0815 Jan 15 '13 at 21:43
  • @sl0815 You can't pass the address of a `malloc()`ed buffer to `mmap()` and expect it to make the mapping at that address. That address is obviously busy: it is already occupied by the `malloc()`ed buffer. You cannot map 2 things to the same address. – Celada Jan 15 '13 at 21:47
  • @Celada: i didn't use mmap in a long time but using non-malloc'd memory is usually a bad idea. so i thought: malloc mem and the let it be used by mmap. what other use could there be for the address hint? but i don't know whether mmap uses some kind of malloc itself for that. mhhh ... didn't think of that possibility – sl0815 Jan 15 '13 at 21:58

2 Answers2

4

The strategy to call mmap() multiple times using MAP_FIXED to specify a fixed address for the second and subsequent mappings should work, but the problem is that if there was anything already mapped into the memory immediately after the first mapping, it will get clobbered, because MAP_FIXED automatically unmaps whatever used to be there before making the new mapping.

I took a look at the layout of some mappings in the address space on a Linux system here, and I observed that, at least some of the time, the addresses chosen by the kernel for memory mappings grow downward from a high address to a low address. That is, a new mapping is given the address space immediately below the address space used by the most recent existing mapping. Under that strategy, when you make your first mapping, it is virtually guaranteed that the address space immediately following that mapping is already occupied by something else (and it's probably something important, too, like a system library). Other systems (different kernel version, different architecture, or non-Linux, etc...) might use different address space allocation strategies that don't make this problem unlikely, but you should assume that it can happen and guard against it by using the following technique.

  1. First make a dummy mapping that is the sum of the size of all of the mappings you want to construct. So if you want to map the first 3 pages of the file, then skip 4, then map three more, make a dummy mapping of 6 pages.

    For this dummy mapping, you can just map anonymous memory (MAP_ANONYMOUS). Thanks to Basile Starynkevitch for the suggestion to also use MAP_NORESERVE for this mapping.

  2. Replace this dummy mapping piece by piece with the mappings of the file you actually want, using MAP_FIXED to specify the precise address you would like each mapping to appear at.

    EDIT: I originally suggested destroying the dummy mapping with munmap() before reusing the address space for new mappings, but thanks to jstine for pointing out that this is unnecessary (and it introduces a race condition if your program is multithreaded).

    For the first mapping, use the start address by the dummy mapping. Calculate the address for the second mapping as the start address of the dummy mapping plus the size of the first mapping. This should place the second mapping right after the end of the first mapping. And so on for the third and fourth mappings. In your scenario, everything is page-sized and page-aligned, so there will be no gaps due to alignment.

After you finish making all of the mappings in step 2, there should be nothing left of the original dummy mapping.

Community
  • 1
  • 1
Celada
  • 21,627
  • 4
  • 64
  • 78
  • And what happens if another program outside your control uses mmap right after you destroy the dummy mapping and accidently chooses this nice free "hole"? I don't believe this could work reliably – sl0815 Jan 15 '13 at 22:09
  • @sl0815: You're mapping to virtual memory, are you not? So other programs won't affect you. (However, I make no comment as to whether this strategy would work reliably...) – Oliver Charlesworth Jan 15 '13 at 22:15
  • I should read those man pages more carefully. @everyone: ignore my stupid comment above. – sl0815 Jan 15 '13 at 22:17
  • Helpful tip: `MAP_FIXED` makes the call to `munmap()` unnecessary. `MAP_FIXED` flag disregards the usual kernel reservation checking, so you can specify it with already-reserved addresses and it will just implicitly clobber whatever memory was previously mapped there. This is part of the posix standard for MAP_FIXED. – jstine Jan 18 '13 at 19:53
  • @jstine turns out that's important, so I'm going to edit the answer with that information. – Celada Jan 18 '13 at 20:14
4

In addition of previous answer by Celada, you might also be interested by the Linux specific remap_file_pages(2) syscall after your mmap(2).

And the first mmap might use MAP_NORESERVE to avoid spending swap space (and just reserve the address space, not the data).

Community
  • 1
  • 1
Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • Thanks for those additional ideas. I didn't know about either `remap_file_pages()` nor `MAP_NORESERVE`. Both very Linux-specific and non-portable, I expect :-) – Celada Jan 17 '13 at 15:37
  • AFAIR, `MAP_NORESERVE` is available on some other systems (Solaris, perhaps). But `remap_file_pages` is indeed very Linux specific. – Basile Starynkevitch Jan 17 '13 at 16:58