substream from istream

Question

Suppose I have an ifstream which represents a large file containing lots of sub-files aggregated together. I want to be able to create a "sub" istream from the larger ifstream (given a size and offest) representing a part of the file so other code can read from that substream as if it was an independent istream.

Any ideas on how I might accomplish this?

EDIT - I would prefer to avoid boost.

score 7 · Accepted Answer · edited Feb 05 '18 at 14:18

This is an example of a streambuf "filter" that reads from a contained streambuf starting at a specified location and reading up to a specified size. You create substreambuf, passing your original streambuf in and substreambuf then translates access so that everything is read from the desired location in the underlying streambuf.

Most of the overhead involved in calling sgetc and snextc from underflow and uflow should optimize away. Many extraction operators work byte by byte, so there should not be additional overhead beyond maintaining the read position within the subsection and checking for the end of the subsection. Of course, reading large chunks of data will be less efficient with this class (although that could be fixed).

This still needs improvements like testing that the requested location is within the underlying streambuf.

class substreambuf : public std::streambuf
{
public:

    substreambuf(std::streambuf *sbuf, std::size_t start, std::size_t len) : m_sbuf(sbuf), m_start(start), m_len(len), m_pos(0)
    {
        std::streampos p = m_sbuf->pubseekpos(start);
        assert(p != std::streampos(-1));
        setbuf(NULL, 0);
    }

protected:

    int underflow()
    {
        if (m_pos + std::streamsize(1) >= m_len)
            return traits_type::eof();
        return m_sbuf->sgetc();
    }

    int uflow()
    {
        if (m_pos + std::streamsize(1) > m_len)
            return traits_type::eof();
        m_pos += std::streamsize(1);
        return m_sbuf->sbumpc();
    }

    std::streampos seekoff(std::streamoff off, std::ios_base::seekdir way, std::ios_base::openmode which = std::ios_base::in | std::ios_base::out)
    {
        std::streampos cursor;

        if (way == std::ios_base::beg)
            cursor = off;
        else if (way == std::ios_base::cur)
            cursor = m_pos + off;
        else if (way == std::ios_base::end)
            cursor = m_len - off;

        if (cursor < 0 || cursor >= m_len)
            return std::streampos(-1);
        m_pos = cursor;
        if (m_sbuf->pubseekpos(m_start + m_pos, std::ios_base::beg) == std::streampos(-1))
            return std::streampos(-1);

        return m_pos;
    }

    std::streampos seekpos(std::streampos sp, std::ios_base::openmode which = std::ios_base::in | std::ios_base::out)
    {
        if (sp < 0 || sp >= m_len)
            return std::streampos(-1);
        m_pos = sp;
        if (m_sbuf->pubseekpos(m_start + m_pos, std::ios_base::beg) == std::streampos(-1))
            return std::streampos(-1);
        return m_pos;
    }

private:
    std::streambuf *m_sbuf;
    std::streampos m_start;
    std::streamsize m_len;
    std::streampos m_pos;
};

It can be used like this

using namespace std;

void somefunc(ifstream &bigifs)
{
    substreambuf sbuf(bigifs.rdbuf(),100,100);
    //new istream with the substreambuf as its streambuf
    istream isub(&sbuf);

    //use isub normally
}

This was inspired by Filtering Streambufs

The `underflow` method should not update `m_read`, should it? — uranix, Aug 21 '13 at 12:27
The code example provided is fraught with issues and should not be considered the correct answer. The `underflow` function should not be incrementing `read`. The `uflow` return value should be `sbuf->sbumpc();`, not `m_sbuf->snextc();`. The code provided also doesn't count `seek` operations towards the `read` value (the usefulness of which is questionable with to begin with). — Colin Basnett, Mar 07 '14 at 05:05
I fixed those issues. Feel free to edit some more if I missed anything else. @cmbasnett, besides the implementation mistakes are you saying this is just the wrong way to do what the poster asked? — BHS, Mar 08 '14 at 07:25
Minor note as I tried to use this code, the implementation is very naive and gives you sub-par performance. The uflow command is called for every single byte read in this version. In this form it is worthless outside of academia. — Lawrence Kok, Feb 05 '18 at 11:21

score 1 · Answer 2 · answered Oct 01 '11 at 22:25

I've done something like this using the Boost.Iostreams library. Look under Tutorial|Writing Devices. The idea is to create a "device" class which implements the low-level interface (read/write/seek) and then instantiate an istream/ostream derived class using your device class to do the actual I/O.

score 1 · Answer 3 · answered Oct 04 '11 at 04:23

All iostreams put most of their custom logic in their streambuf specializations. fstream (or basic_fstream) initializes istream with an instance of file_buf. Same for stringstream (stringbuf). If you want to roll your own substream stream, you can do it by implementing your own streambuf in terms of a parent stream.

score 0 · Answer 4 · answered Oct 04 '11 at 05:42

Just a little idea : If you have control over the client side of the code (i.e. the part that uses the input stream), I suggest you modify it to accept two additional parameters, like illustrated below :

// Old code
void ClassUsingInput::SetInput(std::streambuf & inputbuf)
{
   // Implementation ...
}

Can become :

// New code
void ClassUsingInput::SetInput(std::streambuf & inputbuf, std::streampos position, std::streamsize size) 
{
    inputbuf.pubseekpos(position) ;
    // internally use size to detect end-of-substream
}

substream from istream

4 Answers4

Linked