C++ cereal de-serialization trouble with large size vector

Question

I hope to serialize large size vector with cereal, C++ serialization library.

But, if trying to do that, the exception "Failed to read " + std::to_string(size) + " bytes from input stream! Read " + std::to_string(readSize)" is thrown.

Does anyone know a good solution for this? I'm using VisualStudio 2017.

The source code is shown below.

#include <iostream>
#include <fstream>
#include "include\cereal\cereal.hpp"
#include "include\cereal\archives\binary.hpp"
#include "include\cereal\types\vector.hpp"
#include "include\cereal\types\string.hpp"

void show(std::vector<int> v) {
    for (auto i : v)std::cout << i << ",";
    std::cout << std::endl;
}

int main(void) {

    const std::string file_name = "out.cereal";

    {
        std::vector<int> src;
        // const int STOP = 10;  //OK
        const int STOP = 1000; // NG

        for (int i = 0; i < STOP; i++)src.push_back(i);
        std::cout << "src:" << std::endl;
        show(src);

        std::ofstream ofs(file_name, std::ios::binary);
        cereal::BinaryOutputArchive archive(ofs);
        archive(src);
    }

    {
        std::vector<int> dst;
        std::fstream fs(file_name);
        cereal::BinaryInputArchive iarchive(fs);
        iarchive(dst);
        std::cout << "dst:" << std::endl;
        show(dst);
    }

#ifdef _MSC_VER
    system("pause");
#endif
    return 0;
}

Answer 1

You code works fine for me in Linux, so I think it is to do with the difference between text and binary handling on Windows. Check that you pass std::ios::binary when you are constructing the input stream. Also construct it as std::ifstream rather than just std::fstream.

I think this might have to do with Windows expecting (or adding) a Unicode byte-order mark, which is confusing the serializer.