How can I know the size of Cereal's binary output?

Question

I'm trying to transmit Cereal's serialization result (binary archive) via a TCP link. For the receiving side the first thing is to decide how many bytes (at least) are needed to recover the message object, but seems there is no way to get this information.

The serialization result of a struct with all members of primitive types seems to be as big as the result of sizeof the struct. Is there any guarantee on this? I'm having the concern because if serialized by the portable binary archive, the result is 1 byte bigger. I know this is to record the endianness, but how can the receiving side know this? The only safe method I came up with is to serialize the message and get the size from the output stream. For receiving side this looks not so graceful.

Danath · Answer 1 · 2017-01-09T02:12:19.553

I have found a solution by myself. Shared here in case it's also useful to others.

The most reliable way to get the size is to actually serialize the data, so what's done by me is avoiding the usual memory allocation/access overhead during the serialization (when work with std::stringstream).

The solution begins with deriving from std::basic_streambuf<> (for Cereal it has to be std::streambuf)

class COUNTER_BUFFER : public std::streambuf
{
    private :
        size_t mSize = 0;

    private :
        int_type overflow(int_type C)
        {
            return mSize++;
        }

    public :
        size_t Size(void) const
        {
            return mSize;
        }
};

Usually the class derived from std::basic_streambuf should write the character into some buffer (file, string etc.), but here it counts how many characters (bytes) are written.

Next step assign an instance to std::basic_ostream which is passed to an archive from Cereal:

COUNTER_BUFFER Buffer;
std::basic_ostream<char> Stream(&Buffer);
cereal::BinaryOutputArchive Archive(Stream);

Serialize any data structure normally and when it's done, call Buffer.Size() to get the size.

For data structure which contains only primitive types, there should be a way to get the serialization size at compile time, but with this non-optimal method I can only get it at run time.

It's just good enough to me.

The return value of overflow can overflow, its an int. I think mSize should be an int_type. — jossgray, Jan 26 '17 at 17:05
think this should be safe. `int_type overflow(int_type C) override { D3UNUSED(C); if (m_size == std::numeric_limits::max()) return EOF; // We're gonna overflow so fail. m_size++; return 0; }` — jossgray, Jan 26 '17 at 17:29
Even better solution is to override `xsputn` for better performance. — jossgray, Feb 15 '17 at 22:55

How can I know the size of Cereal's binary output?

1 Answers1