C++ how to write a ifstream for multiple files?

Question

So I have several files that form a contingent number of lines, so let's call them file1, file2, file3 and so forth. What I want to do is to create a istream class which streams all the files as if they were one. One thing i got is not to subclass std::istream but to reimplement streambuf. My question is: how would I go about this - how would i be able to read from multiple files without having them all in memory?

Answer 1

Is it possible to make an istream that reads multiple files?

Yes, but you'll have to make it yourself. You can implement your own istream class by extending std::istream, and then implementing the methods it defines:

class custom_istream : public std::istream {
    std::ifstream underlying; 
    //... implement methods
};

The interface of std::istream is sufficiently flexible to allow you to do what you want. However, implementing std::istream would require a lot of work, as you'd have to implement the entirety of the interface.

Is there an easier solution?

If you only need a subset of the functionality provided by std::istream, you could just write your own class.

For example, if you only need to be able to read lines from the file, the below class will work just fine for multiple files:

class MultiFileReader {
    std::ifstream filestream; 
    std::ios_base::openmode mode;  
    size_t file_index = 0; 
    std::vector<std::string> filenames; 
    void open_next_file() {
        file_index++; 
        filestream = std::ifstream(filenames.at(file_index), mode); 
    }
   public:
    MultiFileReader(std::vector<std::string> const& files, std::ios_base::openmode mode)
      : filestream(files[0], mode), mode(mode) {}
    // Checks if there's no more files to open, and no more to read
    // in the current file
    bool hasMoreToRead() {
        if(file_index == filenames.size()) return false;
        if(file_index + 1 == filenames.size()) 
            return not filestream.eof(); 
        return true;
    }
    std::string read_line() {
        if(not hasMoreToRead()) {
            throw std::logic_error("No more to read"); 
        }
        // If at the end of the file, open the next file
        if(filestream.eof()) {
            open_next_file(); 
        }
        else {
            std::string line; 
            std::getline(filestream, line);
            return line; 
        }
    }
};