Stack Overflow
Stack Overflow

Questions tagged [mmap]

mmap is a POSIX-compliant Unix system call that maps files or devices into memory.

mmap() creates a new mapping in the virtual address space of the calling process.

https://en.wikipedia.org/wiki/Mmap

1936 questions
24
votes
4 answers

Python mmap 'Permission denied' on Linux

I have a really large file I'm trying to open with mmap and its giving me permission denied. I've tried different flags and modes to the os.open but its just not working for me. What am I doing wrong? >>> import os,mmap >>> mfd = os.open('BigFile',…
tMC
  • 18,105
  • 14
  • 62
  • 98
24
votes
2 answers

Resizing numpy.memmap arrays

I'm working with a bunch of large numpy arrays, and as these started to chew up too much memory lately, I wanted to replace them with numpy.memmap instances. The problem is, now and then I have to resize the arrays, and I'd preferably do that…
Michael
  • 7,316
  • 1
  • 37
  • 63
24
votes
3 answers

What’s the purpose of mmap memory protection PROT_NONE

I was reading the documentation of mmap and fell upon this line: PROT_NONE Pages may not be accessed. Is there any use to map a file to memory but never access it?
qdii
  • 12,505
  • 10
  • 59
  • 116
23
votes
1 answer

Mapping an array to a file via Mmap in Go

I'm trying to map an array to a file via Mmap, the array could be any type, like float64. In C, I find this one. After reading some texts, I wrote this sample. I don't know if it is correct, and it is not writing the values to the file. If I…
tfmoraes
  • 696
  • 1
  • 8
  • 18
23
votes
4 answers

mmap( ) vs read( )

I'm writing a bulk ID3 tag editor in C. ID3 tags are usually at the beginning of an mp3 encoded file, although older (version 1) tags are at the end. The app is designed to accept a directory and frame ID list from the command line, then recurse…
J. Andrew Laughlin
  • 1,926
  • 3
  • 21
  • 33
23
votes
3 answers

How to create and write to memory mapped files?

Editor's note: This code example is from a version of Rust prior to 1.0 and the code it uses does not exist in Rust 1.0. Some answers have been updated to answer the core question for newer versions of Rust. I'm trying to create a memory mapped…
mkhq
  • 511
  • 1
  • 4
  • 14
22
votes
1 answer

numpy vs. multiprocessing and mmap

I am using Python's multiprocessing module to process large numpy arrays in parallel. The arrays are memory-mapped using numpy.load(mmap_mode='r') in the master process. After that, multiprocessing.Pool() forks the process (I presume). Everything…
user124114
  • 8,372
  • 11
  • 41
  • 63
22
votes
1 answer

MAP_ANONYMOUS with C99 standard

I have an application that uses the mmap system call, I was having an issue getting it to compile for hours looking as to why I was getting MAP_ANON and MAP_ANONYMOUS were undeclared, I had a smaller section of code that I used and I saw I could…
Jesus Ramos
  • 22,940
  • 10
  • 58
  • 88
22
votes
1 answer

munmap() failure with ENOMEM with private anonymous mapping

I have recently discovered that Linux does not guarantee that memory allocated with mmap can be freed with munmap if this leads to situation when number of VMA (Virtual Memory Area) structures exceed vm.max_map_count. Manpage states this (almost)…
user1143634
21
votes
4 answers

What is the difference between writing to a file and a mapped memory?

I have the following questions related to handling files and mapping them (mmap): We know that if we create a file, and write to that file, then either ways we are writing to the memory. Then why map that file to memory using mmap and then…
RajSanpui
  • 11,556
  • 32
  • 79
  • 146
21
votes
2 answers

When and how is mmap'ed memory swapped in and out?

In my understanding, mmap'ing a file that fits into RAM will be like having the file in memory. Say that we have 16G of RAM, and we first mmap a 10G file that we use for a while. This should be fairly efficient in terms of access. If we then mmap a…
Masseman
  • 359
  • 3
  • 13
21
votes
3 answers

Invalid argument for read-write mmap?

I'm getting -EINVAL for some reason, and it's not clear to me why. Here's where I open and attempt to mmap the file: if ((fd = open(argv[1], O_RDWR)) < 0) { fprintf(stderr, "Failed to open %s: %s\n", argv[1], strerror(errno)); return…
alexgolec
  • 26,898
  • 33
  • 107
  • 159
20
votes
1 answer

vmsplice() and TCP

In the original vmsplice() implementation, it was suggested that if you had a user-land buffer 2x the maximum number of pages that could fit in a pipe, a successful vmsplice() on the second half of the buffer would guarantee that the kernel was done…
user239558
  • 6,964
  • 1
  • 28
  • 35
20
votes
3 answers

How to share APC cache between several PHP processes when running under FastCGI?

I'm currently running several copies of PHP/FastCGI, with APC enabled (under Apache+mod_fastcgi, if that matters). Can I share cache between the processes? How can I check if it's shared already? (I think the apc.mmap_file_mask ini setting might…
mjs
  • 63,493
  • 27
  • 91
  • 122
20
votes
2 answers

How does mmap work?

I am working on programs in Linux which needs mmap file from harddrive, but i have a question, what can make it fail. Like if all the memories are fragmented, which has only 200M each, but i want to mmap a file to a memory of 1000M, will it…
StevenWang
  • 3,625
  • 4
  • 30
  • 40
1 2
3
…
99 100