19

boost::asio's various read and write functions and methods accept boost::asio::buffer. According to buffer's documentation, a mutable std::string cannot be wrapped in boost::asio::buffer, and thus cannot be used for asio's read functions. This is probably due to the fact that std::string does not allow mutable access to its internal buffer (this was discussed previously here).

This is a shame, because std::string is a convenient way to represent mutable buffers of data in C++. Without it, we're either left with POD arrays, boost::array and std::vector<char>. The first two are inconvenient with variable-length messages. std::vector<char> can work, but it's an unnatural way to carry buffers of data around (*)

Questions:

  1. Are there other alternatives to std::string with boost::asio for reading buffers? Am I missing something here?
  2. I wonder why std::vector<char> is supported in a mutable buffer. Is it because it guarantees its internal buffer is contiguous in memory and allows mutable access to it with &vec[0] ?

Thanks in advance


(*) IMHO. Look at protobuf serialization for instance - it offers serialization into std::string but not into std::vector<char>, at least not explicitly.


EDIT: I ended up using vector<char> after all. protobuf allows serialization into a vector<char> by means of the SerializeToArray call which takes a pointer (&vec[0] can be passed there).

Community
  • 1
  • 1
Eli Bendersky
  • 263,248
  • 89
  • 350
  • 412
  • 3
    I normally use `vector` - because I've found this to be faster in all my cases for processing operations on received data. However I have seen uses of uses of `asio::streambuf` as well: http://www.boost.org/doc/libs/1_40_0/doc/html/boost_asio/reference/streambuf.html. – Nim Mar 09 '11 at 15:56
  • @Nim: I considered `asio::streambuf` but it appears problematic with fixed-size reads, and more suitable for `read_until` like in HTTP. But I may be missing something ... – Eli Bendersky Mar 09 '11 at 16:05
  • @Eli: I've always found `std::vector` to be the natural way to express a buffer of bytes. The unknown signedness of `char` is annoying... And semantically a `string` does not really make sense. Don't forget that `protobuf` proposes to serialize to plain text, in which case a `string` does appear natural. – Matthieu M. Mar 09 '11 at 16:08
  • @Mattieu M.: I don't think `protobuf` proposes to serialize to plain text, its serialization output definitely looks like binary data (not ascii) – Eli Bendersky Mar 09 '11 at 16:13
  • 1
    @Eli I'm not sure what you mean by `asio::streambuf` being problematic for fixed size reads. I've used it successfully in several projects, it integrates nicely with Boost.Serialization. Could you edit your question to clarify your concerns? – Sam Miller Mar 09 '11 at 20:41
  • @Sam: I didn't mention `asio::streambuf` in my question originally, indeed because it wasn't 100% clear to me how to use it with fixed size reads and asio. Could you point to an example (or add one as an answer) of showing how to read fixed-length chunks into a `std::sstream`? – Eli Bendersky Mar 10 '11 at 04:26
  • @Eli, the link in my original comment shows how to do a read using the `asio::streambuf`. The only *annoying* aspect of using this buffer is that you have `commit` the count of bytes read before reading from the buffer... – Nim Mar 10 '11 at 08:46
  • @Eli I've added an [answer](http://stackoverflow.com/questions/5248291/alternatives-to-stdstring-to-use-with-boostasio/5276059#5276059). – Sam Miller Mar 11 '11 at 17:14

3 Answers3

8

Mutable access to a string buffer using &str[0] works fine on all known implementations, and the wording of the upcoming C++0x standardizes makes it officially allowed.

Still, I think you're insane to think that a std::vector is an unnatural representation for a variable-length buffer.

Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • Ben, this sounds good, but `asio::buffer` has no constructor taking non-const `std::string`, so how do you pass it into `asio`? Could it be that `asio` doesn't want to currently rely on behavior which isn't strictly standard? – Eli Bendersky Mar 10 '11 at 04:27
  • 1
    Conversion from `std::string` to `asio::buffer`: `asio::buffer(&str[0], str.size())` – Ben Voigt Mar 10 '11 at 04:59
  • I see, but this is not currently standard C++? It's supported by convention and is supposed to be part of C++0X? – Eli Bendersky Mar 10 '11 at 05:04
  • It's currently implementation-dependent, whether the string data exists in a contiguous array. Before adding it to the C++0x standard draft, the committee surveyed all library implementations they could find and not a single one did anything non-contiguous. – Ben Voigt Mar 10 '11 at 05:06
  • @Ben: Do you have a reference to this survey, I'd like to read more about it? – murrekatt Mar 10 '11 at 08:29
  • Wouldn't str.c_str() give you the standard-guaranteed contiguous buffer pointer? – Alan Mar 11 '11 at 18:07
  • 1
    @Alan: There's no guarantee that changes made to the `c_str()` buffer are incorporated into the string, or that the buffer is even writable in the first place. – Ben Voigt Mar 11 '11 at 18:55
  • @murrekatt: Read the comments here: http://herbsutter.com/2008/04/07/cringe-not-vectors-are-guaranteed-to-be-contiguous/ Herb Sutter is one of C++ standard committee members and formerly was the committee chair. – Ben Voigt Mar 11 '11 at 18:58
5

This is to answer Eli's comment

I didn't mention asio::streambuf in my question originally, indeed because it wasn't 100% clear to me how to use it with fixed size reads and asio. Could you point to an example (or add one as an answer) of showing how to read fixed-length chunks into a std::sstream?

Here's a previous answer about using asio::streambuf and Boost.Serialization. The asio documentation also has an example of a synchronous read:

boost::asio::streambuf b;

// reserve 512 bytes in output sequence
boost::asio::streambuf::mutable_buffers_type bufs = b.prepare(512);

size_t n = sock.receive(bufs);

// received data is "committed" from output sequence to input sequence
b.commit(n);

std::istream is(&b);
std::string s;
is >> s;
Community
  • 1
  • 1
Sam Miller
  • 23,808
  • 4
  • 67
  • 87
2

1) alternatives can be found by checking asio::buffer() function, overloads that return mutable_buffers_1. more flexible (but probably sub-optimal) option is asio::streambuf, useful for (async_)read_until.

if you have fixed-size fields protocol you can use array of asio::mutable_buffer. e.g.

using boost::asio;
int i;
short s;
char data[data_size]; // data_size is defined elsewhere
boost::array<asio::mutable_buffer, 3> bufs = {
    asio::buffer(&i, 4), 
    asio::buffer(&s, 2),
    asio::buffer(data, data_size)
};
asio::read(socket, buffer(bufs)); // socket defined elsewhere

2) you already referenced great answer to this question: "How to asynchronously read to std::string using Boost::asio?"

Community
  • 1
  • 1
Andriy Tylychko
  • 15,967
  • 6
  • 64
  • 112
  • Yes, indeed everything you list here was already mentioned in my question (including the fixed-array approach). Thanks for the reassurance, anyway :-) – Eli Bendersky Mar 10 '11 at 04:29