Stack Overflow
Stack Overflow

Questions tagged [mmap]

mmap is a POSIX-compliant Unix system call that maps files or devices into memory.

mmap() creates a new mapping in the virtual address space of the calling process.

https://en.wikipedia.org/wiki/Mmap

1936 questions
10
votes
1 answer

What is the most reliable / portable way to allocate memory at low addresses on 64-bit systems?

I need to allocate large blocks of memory (to be used by my custom allocator) that fall within the first 32GB of virtual address space. I imagine that if I needed, say, 1MB blocks, I could iterate using mmap and MAP_FIXED_NOREPLACE (or VirtualAlloc)…
Aardappel
  • 5,559
  • 1
  • 19
  • 22
10
votes
1 answer

Why does unaligned access to mmap'ed memory sometimes segfault on AMD64?

I have this piece of code which segfaults when run on Ubuntu 14.04 on an AMD64 compatible CPU: #include #include #include int main() { uint32_t sum = 0; uint8_t *buffer = mmap(NULL, 1<<18, PROT_READ, …
kasperd
  • 1,952
  • 1
  • 20
  • 31
10
votes
1 answer

Copying data from a shared-memory-mapped object using sendfile()/fcopyfile()

Is it possible – and if so prudent – to use sendfile() (or its Darwin/BSD cousin fcopyfile()) to shuttle data directly between a shared-memory object and a file? Functions like sendfile() and fcopyfile() can perform all of the mechanistic…
fish2000
  • 4,289
  • 2
  • 37
  • 76
10
votes
3 answers

Java mmap fails on Android with "mmap failed: ENOMEM (Out of memory)"

Memory mapping a large file on Android in Java works good. But when mapping more than ~1.5GB in total even with multiple mapping calls it fails with: mmap failed: ENOMEM (Out of memory) See the full discussion here. Note: It does not fail on a…
Karussell
  • 17,085
  • 16
  • 97
  • 197
10
votes
1 answer

How does numpy handle mmap's over npz files?

I have a case where I would like to open a compressed numpy file using mmap mode, but can't seem to find any documentation about how it will work under the covers. For example, will it decompress the archive in memory and then mmap it? Will it…
Refefer
  • 541
  • 4
  • 11
10
votes
1 answer

How to work around lack of NUL terminator in strings returned from mmap()?

When mmap()ing a text file, like so int fd = open("file.txt", O_RDWR); fstat(fd, &sb) char *text = mmap(0, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0); the file contents are mapped into memory directly, and text it will not contain a NUL-terminator…
mattst88
  • 1,462
  • 13
  • 21
10
votes
4 answers

How portable is mmap?

I've been considering using mmap for file reading, and was wondering how portable that is. I'm developing on a Linux platform, but would like my program to work on Mac OS X and Windows. Can I assume mmap is working on these platforms?
monkeyking
  • 6,670
  • 24
  • 61
  • 81
10
votes
1 answer

Reading a file to string with mmap

I'm trying to read a file to a string using mmap. I was following this example: http://www.lemoda.net/c/mmap-example/index.html My code looks like this unsigned char *f; int size; int main(int argc, char const *argv[]) { struct stat s; const…
arnoapp
  • 2,416
  • 4
  • 38
  • 70
10
votes
3 answers

Zero a large memory mapping with `madvise`

I have the following problem: I allocate a large chunk of memory (multiple GiB) via mmap with MAP_ANONYMOUS. That chunk holds a large hash map which needs to be zeroed every now and then. Not the entire mapping may be used in each round (not every…
Sergey L.
  • 21,822
  • 5
  • 49
  • 75
10
votes
2 answers

What does it take to be durable on Linux?

I'm writing some software to deal with pretty critical data, and need to know what exactly I need to do to achieve durability. Everywhere I look is contradictory information, so I'd appreciate any insight. There are three ways I write to…
Max
  • 2,760
  • 1
  • 28
  • 47
10
votes
1 answer

Referential transparency and mmap in Haskell

I was hoping to use System.INotify and System.IO.MMap together in order to watch for file modifications and then quickly perform diffs for sending patches over a network. However, in the documentation for System.IO.MMap there's a couple of warnings…
Rehno Lindeque
  • 4,236
  • 2
  • 23
  • 31
10
votes
9 answers

Linux/perl mmap performance

I'm trying to optimize handling of large datasets using mmap. A dataset is in the gigabyte range. The idea was to mmap the whole file into memory, allowing multiple processes to work on the dataset concurrently (read-only). It isn't working as…
Marius Kjeldahl
  • 6,830
  • 3
  • 33
  • 37
9
votes
1 answer

Fast resize of a mmap file

I need a copy-free re-size of a very large mmap file while still allowing concurrent access to reader threads. The simple way is to use two MAP_SHARED mappings (grow the file, then create a second mapping that includes the grown region) in the same…
Eloff
  • 20,828
  • 17
  • 83
  • 112
9
votes
3 answers

Does mmap with MAP_NORESERVE reserve physical memory?

The mmap documentation says following about the flag MAP_NORESERVE. Do not reserve swap space for this mapping. When swap space is reserved, one has the guarantee that it is possible to modify the mapping. When swap space is not reserved one…
MetallicPriest
  • 29,191
  • 52
  • 200
  • 356
9
votes
2 answers

Ubuntu 10.04, error in using MAP_HUGETLB with MAP_SHARED

Following is the code that I am using for mmaping a file in ubuntu with hugepages, but this call is failing with error "invalid argument". However, when I do pass MAP_ANON flag with no file descriptor parameter in mmap, then it works. I am not being…
Faraz
  • 131
  • 1
  • 6
1 2 3
…
99 100