9

Following is the code that I am using for mmaping a file in ubuntu with hugepages, but this call is failing with error "invalid argument". However, when I do pass MAP_ANON flag with no file descriptor parameter in mmap, then it works. I am not being able to understand the possible reason behind this.

Secondly, I am not able to understand why file mmaping is allowed with MAP_PRIVATE when this flag itself means that no change will be written back to file. This can always be accomplished using MAP_ANON, or is there something I am missing ?

Can someone help me with these ?

int32_t main(int32_t argc, char** argv) {
int32_t map_length = 16*1024*1024; // 16 MB , huge page size is 2 MB
int32_t protection = PROT_READ | PROT_WRITE;
int32_t flags = MAP_SHARED | MAP_HUGETLB;
int32_t file__ = open("test",O_RDWR|O_CREAT | O_LARGEFILE,s_IRWXU | S_IRGRP | S_IROTH);
if(file__ < 0 ) {
    std::cerr << "Unable to open file\n";
    return -1;
}

if (ftruncate(file__, map_length) < 0) {
    std::cerr
    << "main :: unable to truncate the file\n"
    << "main :: " << strerror(errno) << "\n"
    << "main :: error number is " << errno << "\n";
    return -1;
}
void *addr= mmap(NULL, map_length, protection, flags, file__, 0);
if (addr == MAP_FAILED) {
    perror("mmap");
    return -1;
}
const char* msg = "Hello World\n";
int32_t len = strlen(msg);
memcpy(addr,msg,len);
munmap(addr, map_length);
close(file__);
return 0;
}
Fred Foo
  • 355,277
  • 75
  • 744
  • 836
Faraz
  • 131
  • 1
  • 6
  • `int32_t main` is an error. Per the C++ standard, the return type of `main` and the type of `argc` should both be just `int`. – Fred Foo Jul 04 '11 at 18:41
  • larsmans, does that affect the output of the program ? On my architecture int is same as int32_t, so it shouldn't matter at all. But from standard point of view it is not right, that I agree. (Earlier I made some error in writing this comment) – Faraz Jul 04 '11 at 18:46
  • Faraz: No, he was just nitpicking. – BjoernD Jul 04 '11 at 21:38
  • Why still use 10.04? – Eric Dec 30 '14 at 08:50

2 Answers2

7

Both your questions come down to the same point: Using mmap() you can obtain two kinds of mappings: anonymous memory and files.

Anonymous memory is (as stated in the man page) not backed by any file in the file system. Instead the memory you get back from an MAP_ANON call to mmap() is plain system memory. The main user of this interface is the C library which uses it to obtain backing storage for malloc/free. So, using MAP_ANON is explicitly saying that you don't want to map a file.

File-backed memory kind of blends in a file (or portions of it) into the address space of your application. In this case, the memory content is actually backed by the file's content. Think of the MAP_PRIVATE flag as first allocating memory for the file and then copying the content into this memory. In truth this will not be what the kernel is doing, but let's just pretend.

HUGE_TLB is a feature the kernel provides for anonymous memory (see Documentation/vm/hugetlb‐page.txt as referenced in the mmap() man page). This should be the reason for your mmap() call failing when using HUGETLB for a file. *Edit: not completely correct. There is a RAM file system (hugetlbfs) that does support huge pages. However, huge_tlb mappings won't work on arbitrary files, as I understand the docs.*

For details on how to use HUGE_TLB and the corresponding in-memory file system (hugetlbfs), you might want to consider the following articles on LWN:

BjoernD
  • 4,720
  • 27
  • 32
  • i have written that mmap is failing with error "Invalid argument", that is the string you get when you do strerno(22) – Faraz Jul 05 '11 at 05:53
  • 3
    I went through all documentation before writing this query and there is no clarity anywhere. I will be glad to read something similar to "MAP_HUGETLB works only with MAP_ANON". Secondly, after mounting hugetlbfs we can do file backed mmap with huge pages (on this fs). If indeed MAP_HUGETLB is not supported for file backed mmap, then how are we able to use hugepages for file backed mmap in that fs ? There has to be something which is missing, right ? – Faraz Jul 05 '11 at 06:02
  • 1
    Thanks BjoernD, I really appreciate that you have unearth those articles for me, however I already read all of them before posting here. This is very crucial for me to understand why this is not supported. If there is only some technical issue because of which this cannot be done in a general kernel release, then I can make change in the kernel code for my personal use. I am actually trying to go through the kernel code to understand this, but till now i do not have much success.And if there is anything on why this isn't possible then I will be glad to hear. Thanks a lot for your time & effort – Faraz Jul 05 '11 at 17:38
  • 1
    Documentation/vm/transhuge.txt mentions that transparent huge pages works only with MAP_ANON in current release, that is why the failure. Doing it via hugetlbfs is not a transparent usage. – Faraz Jul 06 '11 at 07:04
1

Adding MAP_PRIVATE to the flags fixed this for me.

Colin Hercus
  • 41
  • 2
  • The mmap page says you can only specify one of MAP_SHARED or MAP_PRIVATE, not both. http://man7.org/linux/man-pages/man2/mmap.2.html – Mark Lakata Apr 27 '17 at 20:58