2

I would like to use boost::beast to read and write to etcd. For starts, I'd like to be able to do these examples with boost beast. They're easily doable with curl. Etcd can be seen as a key/value store. The functions to set/get (put/range in the examples page) are easy to do with the boost::beast client example. No problems there.

But "watch", I don't understand. According to docs, watch is a continuous stream, unlike the others which are sessions that die instantly after retrieving the result. The curl examples show changing the value and a response on-spot while the watch is still active. I should use that same stream to do everything related to that watch, including stopping it.

My question broadly is: How can I implement this in boost::beast?

Let's say that from the client example I submit ioc.run through a thread with 

std::thread t(&std::iocontext::run, &ioc);
t.detach();

Now I have full control of the client in the main thread. Am I supposed to create new http requests and submit them in async_write over the socket object? But then if I do that, I lose the features of boost::beast in wrapping the http header with nice http::request<http::string_body>. Am I supposed to create the headers manually? Or am I supposed to just send json with some kind of line terminator to indicate that a message is over? What does the communication protocol look like?

An example with boost::beast would be great.

The Quantum Physicist
  • 24,987
  • 19
  • 103
  • 189

1 Answers1

1

It looks like etcd uses "long running requests." For this, you want to use http::read_header[1] or http::async_read_header[2] to obtain the response header, and then use http::read_some[3] or http::async_read_some[4] in a loop to read portions of the response body. For this to work cleanly you want to use the http::buffer_body[5] which is designed for this sort of thing. The HTTP Relay example[6] in the documentation demonstrates the use of buffer_body, and can be adapted to process long-running requests.

[1] https://www.boost.org/doc/libs/1_69_0/libs/beast/doc/html/beast/ref/boost__beast__http__read_header/overload2.html

[2] https://www.boost.org/doc/libs/1_69_0/libs/beast/doc/html/beast/ref/boost__beast__http__async_read_header.html

[3] https://www.boost.org/doc/libs/1_69_0/libs/beast/doc/html/beast/ref/boost__beast__http__read_some/overload2.html

{4] https://www.boost.org/doc/libs/1_69_0/libs/beast/doc/html/beast/ref/boost__beast__http__async_read_some.html

[5] https://www.boost.org/doc/libs/1_69_0/libs/beast/doc/html/beast/ref/boost__beast__http__buffer_body.html

[6] https://www.boost.org/doc/libs/1_69_0/libs/beast/doc/html/beast/more_examples/http_relay.html

Vinnie Falco
  • 5,173
  • 28
  • 43
  • Thank you for this Vinnie. It's very helpful. I have one thing I'm stuck at from the http relay example. Is there a way to consume the serializer data and get it to a string? How can I get the data from that example in a FIFO fashion? In the example it writes serialized data immediately to the output socket. I couldn't find a way to convert the serializer data to something like a string and consume it. The function `buffers_to_string` doesn't work for it. – The Quantum Physicist Apr 09 '19 at 21:24
  • I also wanna point out that after retrieving the header, the strings that are exchanged are only json separated by `\r\n`. I was able to verify this with curl. I don't understand why we need a parser for that since no more http headers are present. On one hand I failed to understand how to use the parser with/out a serializer in a FIFO manner to retrieve the json data. On the other hand I tried to `socket_.async_read_some()` directly, because I thought raw data is all I need, and that created an infinite loop of reading with size = 0. I'd appreciate some hints on this. – The Quantum Physicist Apr 09 '19 at 22:44
  • You need a parser because, what if the transfer-encoding is "chunked?" Beast doesn't know ahead of time that your message body will always be plain. Anyway, if you don't want to use the parser, and you know that the body is not chunk-encoded, you could always just read the bytes yourself from the socket. However, note that the dynamic buffer passed to the beast HTTP read algorithm will likely contain a portion of the body. So after the parser indicates that the header is complete, you need to consume any data in this buffer first, then continue reading from the socket. – Vinnie Falco Apr 10 '19 at 16:00
  • If you want the body in a string (I would suggest a `string_view` instead, to avoid unnecessary allocate, copy, and free cycles) then simply construct a string from your pointer and size. It sounds like you are having trouble understanding how buffers work, I would suggest reading https://www.boost.org/doc/libs/master/libs/beast/doc/html/beast/using_io/asio_refresher.html#beast.using_io.asio_refresher.buffers – Vinnie Falco Apr 10 '19 at 16:03
  • I didn't use a `string_view` (which I am now), but this is basically what I was doing yesterday. I was using an `std::copy` to copy the buffer pointer with its size. Tried this with both the parser and the serializer body and the buffer, and both lead to having chunks of the data repeated. I didn't know what chunking meant before, and I just learned it and probably the repetition explains it. However, based on your explanation, the serializer shouldn't have any repetitions when the data is taken from it, but I see that it does here when I take the pointer from the body. [...] – The Quantum Physicist Apr 10 '19 at 17:39
  • Can you please provide an example on how to take the serializer, using a `string_view` to take data from it, then consume the data from the serializer, based on the http relay example, such that no data will be repeated? – The Quantum Physicist Apr 10 '19 at 17:41
  • That's a big answer, and one that will require a new question. – Vinnie Falco Apr 10 '19 at 21:45
  • Actually I asked [another question today earlier](https://stackoverflow.com/questions/55610026/how-can-i-convert-serialized-data-in-boostbeast-to-a-string-so-that-i-could-pr); would be great if you could provide the example there :-) You answered parts of that question here in the discussion, but the general problem is still not solved, which I'm hoping you could solve with the example. – The Quantum Physicist Apr 10 '19 at 21:53