Copy data from fstream to stringstream with no buffer?

Question

Is there anyway I can transfer data from an fstream (a file) to a stringstream (a stream in the memory)?

Currently, I'm using a buffer, but this requires double the memory, because you need to copy the data to a buffer, then copy the buffer to the stringstream, and until you delete the buffer, the data is duplicated in the memory.

std::fstream fWrite(fName,std::ios::binary | std::ios::in | std::ios::out);  
    fWrite.seekg(0,std::ios::end); //Seek to the end  
    int fLen = fWrite.tellg(); //Get length of file  
    fWrite.seekg(0,std::ios::beg); //Seek back to beginning  
    char* fileBuffer = new char[fLen];  
    fWrite.read(fileBuffer,fLen);  
    Write(fileBuffer,fLen); //This writes the buffer to the stringstream  
    delete fileBuffer;`

Does anyone know how I can write a whole file to a stringstream without using an inbetween buffer?

What's the point? Are you trying to improve throughput? You're likely going to need to ditch `fstream` in that case, iostreams are SLOW. Are you trying to decrease your memory footprint? Reading the file in chunks instead of all at once could help with that. — Ben Voigt, Oct 31 '10 at 19:17

score 37 · Answer 1 · answered Oct 31 '10 at 19:19

37

 ifstream f(fName);
 stringstream s;
 if (f) {
     s << f.rdbuf();    
     f.close();
 }

answered Oct 31 '10 at 19:19

pinkfloydx33

11,863
3
46
63

Benjamin Lindley · Accepted Answer · 2010-10-31T19:23:22.897

32

// need to include <algorithm> and <iterator>, and of course <fstream> and <sstream>
ifstream fin("input.txt");
ostringstream sout;
copy(istreambuf_iterator<char>(fin),
     istreambuf_iterator<char>(),
     ostreambuf_iterator<char>(sout));

edited Oct 31 '10 at 19:23

answered Oct 31 '10 at 19:09

Benjamin Lindley

101,917
9
204
274

@Charles -- Nonetheless, I think this is what he intended. He didn't want to allocate a new char array. He wanted to read directly from the fstream object to the stringstream object. – Benjamin Lindley Oct 31 '10 at 19:17
@BenVoigt, you're right - I misread the OP. I thought they were asking how to read directly into a stringstream. – Charles Salvia Oct 31 '10 at 19:17
This seems to be what I was looking for! @PigBen you are right for me not wanting to use a char array. I have a question: does the copy method account for the current writing position of the stringstream? If I use tellp(8), then use copy() will it still work, or will it write from the beginning of the stringstream? EDIT: I tried it out formyself and it seems to work great. Thank you! – Brad Oct 31 '10 at 19:33
2

Still, I like pinkfloydx's solution (made into a complete example by Andre) better. `std::copy` is going to have to move one element at a time, while `operator <<(istream&)` can potentially be much faster. – Ben Voigt Oct 31 '10 at 22:30
@Ben Voigt: So do I. I actually didn't know you could do that so easily. My testing shows a slight difference. On a 50k file, copied 10 times each with both methods, his averaged 1.45 seconds per copy, and mine averaged 1.62 seconds. – Benjamin Lindley Oct 31 '10 at 23:17
Are you serious? Disk-to-memory speed of 35kB/sec? That's so 1990. Coincidentally, earlier today I started on a rewrite of some of my code from ifstream to MapViewOfFile since the profiling I did on Friday showed a huge amount of time spent in fstream. Fully 10% of my runtime was ofstream taking a lock (this when the file is used from a single thread). Simply ludicrous. – Ben Voigt Nov 01 '10 at 00:18

score 10 · Answer 3 · edited Aug 28 '13 at 12:34

In the documentation for ostream, there are several overloads for operator<<. One of them takes a streambuf* and reads all of the streambuffer's contents.

Here is a sample use (compiled and tested):

#include <exception>
#include <iostream>
#include <fstream>
#include <sstream>

int main ( int, char ** )
try
{
        // Will hold file contents.
    std::stringstream contents;

        // Open the file for the shortest time possible.
    { std::ifstream file("/path/to/file", std::ios::binary);

            // Make sure we have something to read.
        if ( !file.is_open() ) {
            throw (std::exception("Could not open file."));
        }

            // Copy contents "as efficiently as possible".
        contents << file.rdbuf();
    }

        // Do something "useful" with the file contents.
    std::cout << contents.rdbuf();
}
catch ( const std::exception& error )
{
    std::cerr << error.what() << std::endl;
    return (EXIT_FAILURE);
}

score 1 · Answer 4 · answered Oct 31 '10 at 19:30

The only way using the C++ standard library is to use a ostrstream instead of stringstream.

You can construct a ostrstream object with your own char buffer, and it will take ownership of the buffer then (so no more copying is needed).

Note however, that the strstream header is deprecated (though its still part of C++03, and most likely, it will always be available on most standard library implementations), and you will get into big troubles if you forget to null-terminate the data supplied to the ostrstream.This also applies to the stream operators, e.g: ostrstreamobject << some_data << std::ends; (std::ends nullterminates the data).

score 1 · Answer 5 · answered Jan 28 '21 at 15:23

1

If you're using Poco, this is simply:

#include <Poco/StreamCopier.h>

ifstream ifs(filename);
string output;
Poco::StreamCopier::copyToString(ifs, output);

answered Jan 28 '21 at 15:23

Roy Shilkrot

3,079
29
25

Copy data from fstream to stringstream with no buffer?

5 Answers5

Linked