22

I am using an API that takes a FILE * and am using that to create a data buffer in memory:

std::shared_ptr<FILE> f(tmpfile(), fclose);
write_to_file(f.get());
rewind(f.get());
auto data = make_file_buffer(f.get());
return data;

This works, but is slower than writing to a memory buffer.

Is it possible to get this to write to a memory file and avoid reading/writing to disk (like stdin/stdout/stderr read/write to the console)?

NOTE: I am using Linux, so have access to Linux and POSIX APIs.

Nikos C.
  • 50,738
  • 9
  • 71
  • 96
reece
  • 7,945
  • 1
  • 26
  • 28
  • 2
    You can open a file in /dev/shm on Linux, which exists purely in memory. – slugonamission May 29 '13 at 13:53
  • 2
    This is not C, you are using C++ constructs. – wildplasser May 29 '13 at 13:57
  • 1
    @wildplasser The API I am using -- `FILE *`, `tmpfile`, `fclose`, `rewind` (along with the `fputc`, `fwrite`, etc. calls in `write_to_file`) -- are C APIs defined in `stdio.h`. This is why I tagged it as `C` as I was after a C function that returned a `FILE *` to a memory buffer. I used C++ constructs to show how I was using the C API. – reece May 29 '13 at 14:07

2 Answers2

36

Yes, this is possible, see fmemopen.

Damon
  • 67,688
  • 20
  • 135
  • 185
  • 2
    I never actually knew about that. +1 :). – slugonamission May 29 '13 at 13:57
  • 1
    @slugonamission lol, me neither. And it sounds like a useful thing. +1. – Nikos C. May 29 '13 at 14:04
  • 1
    There is also `open_memstream` from that link that allocates the buffer for you and gives you its location and size. Thanks for the answer. – reece May 29 '13 at 14:10
  • How would one do this on Windows? – Arya Pourtabatabaie Apr 04 '18 at 19:34
  • 3
    @AryaPourtabatabaie: Windows, or rather the MSVCRT, does not natively support this. You could probably work around by either opening a file with "temporary" hint (which, unless you run out of physical memory, is mostly equivalent to an in-memory file) or by creating an anonymous file mapping (which, too, is a kind of "memory buffer"), then call `_open_osfhandle()` on the `HANDLE` followed by calling `_fdopen()`. That gives you a `FILE*`. – Damon Apr 05 '18 at 07:34
2

I wrote a comment above, but I'll flesh it out for an answer.

Since your API expects a FILE handle, the easiest way would be to create a file on an in-memory filesystem, such as something mounted with ramfs or tmpfs on Linux.

There's one of these created by default on most Linux systems under /dev/shm, so creating any file under there will still exist as an actual file, but will only exist in memory. Of course, if you reboot, the contents of the file will be lost. I also don't think it can be paged to disk, so try and avoid writing huge files or you'll take up all the space in your memory.

slugonamission
  • 9,562
  • 1
  • 34
  • 41