boost::asio::streambuf shrink-to-fit?

Question

The boost::asio::streambuf size will keep on increasing until consume() is called.
Even after consume() is called, the memory used by the underlying buffer will never be released.

For example: the following code first created a streambuf without specifying the max_size. Then it dump 14Mb data into the streambuf. Then it consume all those 14MB data. At point 2000, the streambuf.size() is 0, but "top" shows the process still take 14MB ram.

I don't want to specify the max_size. Is there anyway to shrink the streambuf after it is empty?

#include <boost/asio.hpp>
#include <iostream>
#include <string>

int main()
{
    {
        boost::asio::streambuf b;
        std::ostream os(&b);

        for(int i= 0; i<1000000; ++i)
        {
            os << "Hello, World!\n";
        }
        std::cout<<"point 1000"<<std::endl;
        std::cout<<"size="<<b.size()<<std::endl;
        // at this point, the streambuf's size is close to 14MB.


        b.consume(b.size());
        std::cout<<"point 2000"<<std::endl;
        std::cout<<"size="<<b.size()<<std::endl;
        // at this point, the streambuf.size() is 0
        // but the streambuf's underlying buffer, which is a vector I assume,
        // still hold that 14MB space.
        // "top" shows the process size is 14M


    }

    // at this point, "top" showed the process size shrinks to almost zero
    std::cout<<"point 4000"<<std::endl;
    return 0;
}

This is very similar to http://stackoverflow.com/questions/9552785/boost-asio-streambuf-dont-release-memory-after-calling-consume but it doesn't indicate a positive answer to your question. — Steve, Nov 06 '12 at 04:59

score 2 · Accepted Answer · answered Nov 06 '12 at 08:03

So, I looked in the sources and two observations appears to me.

You are right, buffer is implemented as std::vector, but this member is private and hasn't any accessors to it.
consume method doesn't touch this field.

For your case, you should destroy buffer object after consume. It's the easiest way to address problem. For optimization purposes, you could remember last buffer size and create new buffer with given capacity by passing last size to constructor. You could write a wrapper around this functionality to achieve readable code.

Thanks for the confirmation. For my case, destruct and reconstruct is not acceptable, that will be even worse than set the max_size. Well if there is no way to shrink it, guess I'll have to set a max_size on it. — John Crane, Nov 06 '12 at 18:27

boost::asio::streambuf shrink-to-fit?

1 Answers1