Reading a file to string with mmap

Question

I'm trying to read a file to a string using mmap.

I was following this example: http://www.lemoda.net/c/mmap-example/index.html

My code looks like this

unsigned char *f;
int size;
int main(int argc, char const *argv[])
{
    struct stat s;
    const char * file_name = argv[1];
    int fd = open (argv[1], O_RDONLY);

    /* Get the size of the file. */
    int status = fstat (fd, & s);
    size = s.st_size;

    f = (char *) mmap (0, size, PROT_READ, 0, fd, 0);
    for (i = 0; i < size; i++) {
        char c;

        c = f[i];
        putchar(c);
    }

    return 0;
}

But I always receive a segemation fault when accessing f[i]. What am I doing wrong?

You do not check the return values of `open` and `mmap`. What happens if you do? — Jongware, Dec 08 '13 at 23:39

mcleod_ideafix · Accepted Answer · 2013-12-09T12:56:14.017

strace is your friend here:

$ strace ./mmap-example mmap-example.c

...
... (lots of output)
...
open("mmap-example.c", O_RDONLY)        = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=582, ...}) = 0
mmap(NULL, 582, PROT_READ, MAP_FILE, 3, 0) = -1 EINVAL (Invalid argument)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++

The mmap man page tells you all you need to know ;)

EINVAL We don't like addr, length, or offset (e.g., they are too large, or not aligned on a page boundary).
EINVAL (since Linux 2.6.12) length was 0.
EINVAL flags contained neither MAP_PRIVATE or MAP_SHARED, or
contained both of these values.

The -EINVAL error is caused by flags that cannot be 0. Either MAP_PRIVATE or MAP_SHARED has to be picked. I have been able to make it work by using MAP_PRIVATE on Linux, x86-64.

So, you have just to add MAP_PRIVATE to mmap():

#include <stdio.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <sys/io.h>
#include <sys/mman.h>

int main(int argc, char const *argv[])
{
    unsigned char *f;
    int size;
    struct stat s;
    const char * file_name = argv[1];
    int fd = open (argv[1], O_RDONLY);

    /* Get the size of the file. */
    int status = fstat (fd, & s);
    size = s.st_size;

    f = (char *) mmap (0, size, PROT_READ, MAP_PRIVATE, fd, 0);
    for (int i = 0; i < size; i++) {
        char c;

        c = f[i];
        putchar(c);
    }

    return 0;
}

NOTE: My first answer did include another possible cause for the EINVAL:

size must be an integral multiple of the page size of the system. To obtain the page size use the function getpagesize().

This is not actually required, but you must take into account that either way, mapping will be always performed in multiples of the system page size, so if you'd like to calculate how much memory is actually been available through the returned pointer, update size as this:

int pagesize = getpagesize();
size = s.st_size;
size += pagesize-(size%pagesize);

Or even just an ordinary debugger would tell you that `f` is `NULL` on the crashing line, implying that the call to `mmap` failed. — Adam Rosenfield, Dec 09 '13 at 00:20
`size` doesn't need to be a multiple of the page size; I think OP's issue was that either `MAP_PRIVATE` of `MAP_SHARED` needs to be specified. From http://pubs.opengroup.org/onlinepubs/7908799/xsh/mmap.html: "The implementation performs mapping operations over whole pages. Thus, while the argument len need not meet a size or alignment constraint, the implementation will include, in any mapping operation, any partial page specified by the range `[pa, pa + len)`." — Adam Rosenfield, Dec 09 '13 at 02:49
You are right: I misread the information about what have to be aligned to page boundary. This is what the man page of mmap says about it: EINVAL We don't like addr, length, or offset (e.g., they are too large, or not aligned on a page boundary). EINVAL (since Linux 2.6.12) length was 0. EINVAL **flags contained neither MAP_PRIVATE or MAP_SHARED**, or contained both of these values. I'll update the answer :) — mcleod_ideafix, Dec 09 '13 at 12:47
`offset must be a multiple of the page size as returned by sysconf(_SC_PAGE_SIZE).`. I use the bit operation to implement. `pa_offset = offset & ~(sysconf(_SC_PAGE_SIZE) - 1);` — firelyu, Feb 22 '17 at 07:54
The example by mcleod_ideafix works well with 1GB file - however when I try with a 4GB file it does nothing. What am i missing? — affluentbarnburner, Jul 29 '23 at 21:13

Reading a file to string with mmap

1 Answers1

Linked