C++ FILE without writing to disk

Question

I'm using a library that has quite a few functions that write to a FILE but none that seem to conveniently dump the same data to an object in memory. Is there any way to create a FILE object (or override it) that stores the data in memory instead of writing to disk -- I'd like to avoid the performance hit of opening/writing/reading from files over and over again.

UPDATE: per Rob's suggestion, trying stringstream:

ss.put(c);

std::string myval = ss.str();

printf("Value: %s\n after writing: %i length %lu\n",myval.c_str(),c, myval.length());

But, now trying to get the data (binary) out of the stringstream has me stuck -- how do I grab the binary data I've been adding?

Only for completeness sake, I mention [`std::stringstream`](http://www.cplusplus.com/reference/iostream/stringstream/). `stringstream` won't solve OP's problem, because he is tied to `FILE`. If he could modify his library, he could use `stringstream`. — Robᵩ, May 02 '11 at 17:56
@Rob - my goal was to avoid modifying the source of the library (to avoid conflict when upgrading, etc), but I do have the option. I think I may take this path, since the other solutions are looking more low-level than my C skills right now. Basically, I need to replace or augment a function that right now is using `fputc` -- I'll be using `put` from `stringstream` instead, right? — jnpdx, May 02 '11 at 18:28
Don't be afraid to use `fmemopen` or `open_memstream`. They look like perfect solutions for you. But, if you do want to use `stringstream`, see my answer. — Robᵩ, May 02 '11 at 19:04
neither function is available to me on iOS, so I'd be stuck doing `mmap` stuff -- another thing I'm not comfortable with yet! — jnpdx, May 02 '11 at 19:28

Cubbi · Accepted Answer · 2012-06-25T16:26:13.323

11

Beside the already mentioned GNU's fmemopen(), which is known in POSIX as open_memstream~~, similar solution can be obtained combining mmap() (using MAP_ANONYMOUS) or any other OS-specific function that returns a file descriptor to a block of memory, and fdopen().~~

EDIT: that was wrong, mmap doesn't create a file descriptor.

edited Jun 25 '12 at 16:26

answered May 02 '11 at 17:43

Cubbi

46,567
13
103
169

Great answer. My limited C skills are making me wary of getting in this deep, though -- see my comment to Rob on the original post. May be able to get away with just a small modification to the original library. – jnpdx May 02 '11 at 18:31
How did you get a file descriptor from mmap that could be used with fdopen? I'm trying to get a FILE pointer to a mmap'd location in memory that doesn't have a backing file. – featherless Jun 25 '12 at 01:12
@featherless You're right, `mmap` doesn't create a file descriptor. I was thinking of `shm_open` when writing that, but `shm_open`'s file descriptor isn't allowed in `fdopen`. Thank you for pointing out the error! – Cubbi Jun 25 '12 at 16:24

Ernest Friedman-Hill · Answer 2 · 2011-05-02T17:51:43.063

5

The GNU libc has, e.g., fmemopen which will give you a FILE * that writes to memory. Try man fmemopen on your Linux system for details.

I suspect (but do not know for sure) that fmemopen is a wrapper that orchestrates the mmap/fdopen approach mentioned by @Cubbi.

edited May 02 '11 at 17:51

answered May 02 '11 at 17:40

Ernest Friedman-Hill

80,601
10
150
186

score 2 · Answer 3 · answered Jun 26 '12 at 19:26

2

If you are on Mac OS X or iOS you don't have access to fmemopen. I've open sourced a solution here:

http://jverkoey.github.com/fmemopen/

answered Jun 26 '12 at 19:26

featherless

2,118
19
19

score 1 · Answer 4 · answered May 02 '11 at 19:03

1

If you have the option of modifying your library, you could use C++ streams instead of C FILE streams.

If your old library function looked like this:

void SomeFun(int this, int that, FILE* logger) {
  ... other code ...
  fprintf(logger, "%d, %d\n", this, that);
  fputs("Warning Message!", logger);
  char c = '\n';
  fputc(c, logger);
}

you might replace that code with:

void SomeFun(int this, int that, std::ostream& logger) {
  ... other code ...
  logger << this << ", " << that << "\n";
  // or: logger << boost::format("%d %d\n") %this %that;
  logger << "Warning Message!";
  char c = '\n';
  logger.put(c);
  // or: logger << c;
}

Then, in your non-library code, do something like:

#include <sstream>    
std::stringstream logStream;
SomeFun(42, 56, logStream);
DisplayCStringOnGui(logStream.str().c_str());

answered May 02 '11 at 19:03

Robᵩ

163,533
20
239
308

EDIT: put code in original question for better formatting I've got it working, up to a basic point. The problem I'm hitting now is getting the data out -- it's binary data. Right now, I have: `ss.put(c); std::string myval = ss.str(); printf("Value: %s\n after writing: %i length %lu\n",myval.c_str(),c, myval.length());` The length() call gives me a value the increases accurately, but I'm stuck trying to get the data out -- the c_str gives me a `const char*` with a length of 4 -- missing most of the binary data – jnpdx May 02 '11 at 19:31
1

If the data that the library returns is binary (that is, potentially has `'\0'` characters embedded in it), then you won't be able to use `printf("%s", ...)` to access it. Are you just trying t display that data? Or do you need to pass it to another function? You can access the data byte-by-byte like this: `myval[i]`. Or you can pass a pointer to the data like this: `MyFunc(myval.data(), myval.length())`. – Robᵩ May 02 '11 at 19:39

score 0 · Answer 5 · answered Jul 22 '19 at 15:24

0

https://github.com/Snaipe/fmem appears to be a portable fmemopen in C. It gives you FILE you can write to and when you done you get a void* that points to the memory where you data is.

answered Jul 22 '19 at 15:24

saraedum

1,389
1
11
7

score 0 · Answer 6 · answered May 02 '11 at 17:43

0

Consider mounting a tmpfs and have the application write to it. Of course this is *nix only.

answered May 02 '11 at 17:43

Alexander Gessler

45,603
7
82
122

C++ FILE without writing to disk

6 Answers6

Linked