Serialization of non default constructible and std::reference wrapper alternatives

Question

I tried serialising my (neural) network and am currently stuck-ish. The issue seems to be that you can't serialize a std::reference_wrapper. I am unsure whether I should either change the way the references to the upper nodes are stored or come up with a way to serialize those.

Are there alternatives to reference_wrappers, which I neglected and still avoid c style pointers? (which are to be avoided as far as i know)

#include <iostream>
#include <fstream>
#include <list>
#include <vector>
#include <functional>

#include <boost/archive/tmpdir.hpp>

#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>

#include <boost/serialization/base_object.hpp>
#include <boost/serialization/utility.hpp>
#include <boost/serialization/list.hpp>
#include <boost/serialization/assume_abstract.hpp>
#include <boost/serialization/shared_ptr.hpp>
#include <boost/serialization/vector.hpp>

typedef float nodeValueType;
typedef std::pair<std::vector<nodeValueType>, std::vector<nodeValueType>> Example;
typedef std::list<Example> ExampleList;

class Node;

class Link
{
public:
Link() = delete;
Link(Node& upperNode, Node& lowerNode)
: Link(upperNode, lowerNode, 1.0e-3f * (std::rand() / (nodeValueType) RAND_MAX))
{
}
Link(Node& upperNode, Node& lowerNode, nodeValueType weight)
: weight_(weight), upperNode_(upperNode), lowerNode_(lowerNode)
{
}
Link(const Link&) = delete;
Link& operator=(const Link&) = delete;

nodeValueType weight_;
std::reference_wrapper<Node> upperNode_;
std::reference_wrapper<Node> lowerNode_;


friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned int version){
ar & this->weight_;
ar & this->upperNode_;
ar & this->lowerNode_;
}
};

class Node
{
public:
Node();

// making it hard to copy us since we really never want to move.
// we are referred in loads of pointers
// therefore moving us invalidates all of them TODO invalidation scheme?
Node(const Node&) = delete;
Node& operator=(const Node&) = delete;

void linkTo(Node& other)
{
assert(this->lowerLinks_.max_size() > (this->lowerLinks_.size() + 1) * 2);
// Link creation
this->lowerLinks_.push_back(std::shared_ptr<Link>(new Link(*this, other)));
other.upperLinks_.push_back(std::shared_ptr<Link>(this->lowerLinks_.back()));
}

std::vector<std::shared_ptr<Link>> lowerLinks_;
std::vector<std::shared_ptr<Link>> upperLinks_;
// serialization
friend class boost::serialization::access;
template<class Archive>
void serialize(Archive& ar, const unsigned int version){
ar & this->lowerLinks_;
}
};

int main()
{
Node n1;
Node n2;

n2.linkTo(n1);

std::string filename(boost::archive::tmpdir());
filename += "/demofile.txt";

std::ofstream ofs(filename);
boost::archive::text_oarchive oa(ofs);
oa << n1 << n2;

Node n3,n4;

// open the archive
std::ifstream ifs(filename);
boost::archive::text_iarchive ia(ifs);

// restore the schedule from the archive
ia >> n3 >> n4;
return 0;
}

Raw owning pointers have to be avoided, regular pointers might be fine. — Jarod42, Mar 15 '18 at 13:54

score 0 · Answer 1 · answered Mar 15 '18 at 14:25

0

There are 2 problems to overcome.

The first is that std::reference_wrapper cannot be null, so you might want to consider a 'nullable reference' type:

template<class T> 
struct NullableReference
{
    constexpr NullableReference() : ptr_(nullptr) {}
    constexpr NullableReference(T& ref) : ptr_(std::addressof(ref)) {}

    constexpr auto operator*() const -> T& { assert(!empty()); return *ptr_; }
    constexpr operator T&() const { assert(!empty()); return *ptr_; }

    constexpr bool empty() const { return !ptr_; }

    template<class Archive>
    void serialize(Archive& ar, unsigned int version)
    {
        ar & ptr_;
    }

private:
    T* ptr_;
};

The other is that Link does not have a default constructor (no doubt because of the non-nullable reference wrappers).

For this reason you may want to consider custom handling of the constructor when deserialising a Link (this is covered in the boost docs IIRC).

Of course, now that you're using a NullableReference, you can allow the default constructor for Link:

...

Link() : weight_(), upperNode_(), lowerNode_() {};

nodeValueType weight_;
NullableReference<Node> upperNode_;
NullableReference<Node> lowerNode_;

....

answered Mar 15 '18 at 14:25

Richard Hodges

68,278
7
90
142

I really do want Link to stay non default constructable since a link shouldn't exist if it doesn't have 2 nodes. Therefore I'll implement save and load functions (if that was what you meant) – Gladaed Mar 15 '18 at 16:53
1

@Gladaed implementation of serialize for non-default constructors here: http://www.boost.org/doc/libs/1_66_0/libs/serialization/doc/serialization.html#constructors – Richard Hodges Mar 15 '18 at 17:17
Can you explain, what the difference between serialize and save_construct_data is? i couldn't find an explanation – Gladaed Mar 16 '18 at 13:03
in another sense when would i ever want to enter code into serialize when i use save/load const data? Only iff. the constructor doesn't mutate the values of membervariables? Why would there ever not be a constructor for that? – Gladaed Mar 16 '18 at 13:34
There might be constructor for the deserialization, but it might not be the most efficient (e.g. cannot be in-place so requires copying) and the converse doesn't apply: you can't "just use the destructor" for serialization either. Simply put: deserialization isn't not construction. You deserialize _into_ an existing object (even if that object is a pointer). – sehe Mar 18 '18 at 01:06

Serialization of non default constructible and std::reference wrapper alternatives

1 Answers1