NgHttp2 invokes request data handler twice, for one single request

Question

I am running a HTTP2 server by using nghttp2. I am trying to figure out why the on_data handler in the server.cpp is called twice. I know it's called twice because when I send a request that contains data, I get the following log output for a single request in the server.

1
2

But if I don't send data in the request. The log output for single request is what I would expect

I send data to the server by using the client and with this line of code

auto req = sess.submit(ec, "GET", "http://127.0.0.1:3000/", "aaaaaaaaaa");

To not send data I just remove the third parameter.

server.cpp

Compiled with g++ server.cpp -o server.out -lnghttp2_asio -lboost_system -lcrypto -lpthread -lssl -lboost_thread

#include <iostream>
#include <nghttp2/asio_http2_server.h>

using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::server;

int main() {
    std::string hostname = "127.0.0.1";
    std::string port = "3000";
    boost::system::error_code ec;
    http2 server;

    int count = 0;
    server.handle("/", [&count](const request &req, const response &res) {
        req.on_data([&res, &count](const uint8_t *data, std::size_t len) {
            std::cerr << ++count << '\n';
            res.write_head(200);
            res.end("done");
        });
    });

    if (server.listen_and_serve(ec, hostname, port)) {
        std::cerr << "error: " << ec.message() << std::endl;
    }
}

Compiled with g++ client.cpp -o client.out -lnghttp2_asio -lboost_system -lcrypto -lpthread -lssl -lboost_thread

#include <iostream>

#include <nghttp2/asio_http2_client.h>

using boost::asio::ip::tcp;

using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::client;

int main(int argc, char *argv[]) {
    boost::system::error_code ec;
    boost::asio::io_service io_service;

    // connect to localhost:3000
    session sess(io_service, "127.0.0.1", "3000");

    sess.on_connect([&sess](tcp::resolver::iterator endpoint_it) {
        boost::system::error_code ec;

        auto req = sess.submit(ec, "GET", "http://127.0.0.1:3000/", "aaaaaaaaaa");
        req->on_response([](const response &res) {
            // print status code and response header fields.
            std::cerr << "HTTP/2 " << res.status_code() << std::endl;
            for (auto &kv : res.header()) {
                std::cerr << kv.first << ": " << kv.second.value << "\n";
            }
            std::cerr << std::endl;

            res.on_data([](const uint8_t *data, std::size_t len) {
                std::cerr.write(reinterpret_cast<const char *>(data), len);
                std::cerr << std::endl;
            });
        });
        req->on_close([&sess](uint32_t error_code) {
            // shutdown session after first request was done.
            sess.shutdown();
        });
    });

    sess.on_error([](const boost::system::error_code &ec) {
        std::cerr << "error: " << ec.message() << std::endl;
    });

    io_service.run();
}

score 0 · Accepted Answer · answered Aug 26 '21 at 22:05

If you print out also the length of the data frames, there would be more information to reason about.

My guess is that the client sends a first DATA frame with length 10 for aaaaaaaaaa and end_stream=false, and a second DATA frame with length 0 and end_stream=true, to signal the end of the stream on the request side.

For the case where you don't send content, the client is probably sending just a DATA frame of length 0 with end_stream=true.

I imagine this can be optimized by using a different API (different method on sess or different parameters) so that you can make it more efficient.

The first case, a request with content, would ideally have a single DATA frame of length 10 and end_stream=true, while the second case (no request content), would have just a HEADERS frame with end_stream=true and no DATA frame at all.

score 0 · Answer 2 · answered Jul 10 '22 at 02:57

When request body is received, on_data primitive is called N times, until all the data received on nghttp2 reactor is consumed.

Take a look to this gist:

As you can see, I'm using algorithm std::copy and stringstream to append the data chunks received when length is positive. If you send, for example, 806 bytes, on_data could be called once with len=806, or perhaps twice with len=8 and then len=798.

Finally, on_data is called with len=0 and then you could process the complete transaction.

NgHttp2 invokes request data handler twice, for one single request

2 Answers2