Assign string to zmq::message_t without copying

Question

I need to do some high performance c++ stuff and that is why I need to avoid copying data whenever possible.

Therefore I want to directly assign a string buffer to a zmq::message_t object without copying it. But there seems to be some deallocation of the string which avoids successful sending.

Here is the piece of code:

for (pair<int, string> msg : l) {
    comm_out.send_int(msg.first);
    comm_out.send_int(t_id);

    int size = msg.second.size();
    zmq::message_t m((void *) std::move(msg.second).data(), size, NULL, NULL);
    comm_out.send_frame_msg(m, false); // some zmq-wrapper class
}

How can I avoid that the string is deallocated before the message is send out? And when is the string deallocated exactly?

Regards

score 1 · Accepted Answer · answered Aug 06 '13 at 10:17

1

I think that zmq::message_t m((void *) std::move(msg.second).data()... is probably undefined behaviour, but is certainly the cause of your problem. In this instance, std::move isn't doing what I suspect you think it does.

The call to std::move is effectively creating an anonymous temporary of a string, moving the contents of msg.second into it, then passing a pointer to that temporary data into the message_t constructor. The 0MQ code assumes that pointer is valid, but the temporary object is destroyed after the constructor of message_t completes - i.e. before you call send_frame.

Zero-copy is a complicated matter in 0mq (see the 0MQ Guide) for more details, but you have to ensure that the data that hasn't been copied is valid until 0MQ tells you explicitly that it's finished with it.

Using C++ strings in this situation is hard, and requires a lot of thought. Your question about how to "avoid that the string is deallocated..." goes right to the heart of the issue. The only answer to that is "with great care".

In short, are you sure you need zero-copy at all?

answered Aug 06 '13 at 10:17

SteveLove

3,137
15
17

Dear SteveL, zero-copy is always nicer for high-performance. If it is possible to avoid a copy, it should be done. Don't you think so? In total that are 33MB that have to be copied. My second way to realize this would be to extend the zmq.hpp by another constructor that takes a string&& and that applies that string to a class-member string. What do you think about this idea? Regards – moo Aug 06 '13 at 10:30
@user2237976 - zero-copy can make code hard to write correctly and hard to read, so doing it always isn't necessarily the right thing. I wouldn't recommend changing the `zmq.hpp` code either - that's a bit of a maintenance issue when you want to upgrade - but the approach sounds possible. Perhaps you could wrap `zmqcpp` in your own abstraction, or indeed wrap the native 0MQ C library with the changes you need. – SteveLove Aug 06 '13 at 10:41
As this code is just a research thing, it is not necessary that is very readable or compatible to any library. I will check whether my approach works and tell you here. – moo Aug 06 '13 at 11:00

Assign string to zmq::message_t without copying

1 Answers1