How to connect with boost::asio to a HTTPS server using a proxy?

Question

In our application we use boost::asio to connect using HTTP and HTTPS. We also can use a HTTP proxy. Now i need to add support for a HTTPS server using a proxy.

I studied quite a few samples and find that the needed steps seem to be:

1. Create a HTTP Connection to the Proxy
2. Send CONNECT myhost.com:443 to the Proxy
3. Then continue using the connection as a SSL tunnel

The problem i am facing lies in STEP 3. I can EITHER connect using unencrypted HTTP OR connect using SSL/HTTPS. If i use a HTTPS connection before the handshake (in order to send CONNECT) that fails as well as performing a SSL handshake for a plain HTTP connection.

This post here contains some fragments - but it does not contain the step i am missing: Connect SSL server via HTTP proxy using boost

Any hints what i am missing?

Sample code:

using boost::asio::ip::tcp;
namespace ssl = boost::asio::ssl;
typedef ssl::stream<tcp::socket> ssl_socket;

// Create a context that uses the default paths for
// finding CA certificates.
ssl::context ctx(ssl::context::sslv23);
ctx.set_default_verify_paths();


// Open a socket and connect it to the remote host.
boost::asio::io_context io_context;
ssl_socket socket(io_context, ctx);
boost::asio::connect(socket.lowest_layer(), resolver.resolve(query));
socket.lowest_layer().set_option(tcp::no_delay(true));
socket.set_verify_callback(ssl::host_name_verification(...));
boost::system::error_code error = boost::asio::error::host_not_found;

boost::asio::streambuf request2;
std::ostream request_stream2(&request2);

boost::asio::streambuf response2;

request_stream2 << "CONNECT " << in_server << ":443 HTTP/1.0\r\n";
request_stream2 << "Host: " << in_server << ":443 \r\n";
AddBasicUserAuthHeader(request_stream2, testUrl);
request_stream2 << "Proxy-Connection: keep-alive\r\n";
request_stream2 << "Connection: keep-alive\r\n\r\n";

// Send the request - this will fail with "write: uninitialized"
boost::asio::write(socket, request);

... wait and process response
socket.set_verify_mode(ssl::verify_none);
socket.handshake(ssl_socket::client);

This code fails in boost::asio::write with "write: uninitialized". I can not figure out how to use the connection as plain HTTP/TCP at this point. The other way round - first create a plain HTTP connection fails when trying to switch to HTTPS.

Answer 1

Oof. That's some spotty code. Cleaning up the missing stuff and misspelled stuff, I noticed:

• you might need to explicitly or implicitly flush the ostream (just good practice really)

• you're writing to an ssl stream before the handshake? Just write to the underlying socket if you wanted:

   boost::asio::write(socket, request);
  

  Should be

   boost::asio::write(socket.next_layer(), request);
  

That said, I'd probably not construct the ssl stream until you have a proxy response. I'd also write the request using Beast to rule out any mishaps in dealing with the implementation details manually.

---

Live Demo

#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <boost/beast.hpp>
#include <boost/beast/ssl.hpp>
#include <iostream>

using boost::asio::ip::tcp;
namespace ssl      = boost::asio::ssl;
namespace beast    = boost::beast;
namespace http     = beast::http;
using ssl_socket   = ssl::stream<tcp::socket>;
using Request      = http::request<http::empty_body>;
using Response     = http::response<http::empty_body>;
using BodyResponse = http::response<http::string_body>;

int main()
{
    std::string const in_server = "example.com";
    boost::asio::io_context io_context;
    tcp::resolver           resolver(io_context);
    tcp::resolver::query    query{"localhost", "8888"};

    // Create a context that uses the default paths for
    // finding CA certificates.
    ssl::context ctx(ssl::context::sslv23);
    ctx.set_default_verify_paths();

    // Open a socket and connect it to the remote host.
    ssl_socket socket(io_context, ctx);
    boost::asio::connect(socket.lowest_layer(), resolver.resolve(query));
    socket.lowest_layer().set_option(tcp::no_delay(true));
    socket.set_verify_callback(ssl::host_name_verification(in_server));

    {
        Request request(http::verb::connect, in_server + ":443", 10);

        request.set(http::field::host, in_server+":443");
        request.set(http::field::proxy_connection, "keep-alive");
        request.set(http::field::connection, "keep-alive");
        request.set(http::field::proxy_authorization, "basic aGVsbG86d29ybGQ=");
        request.prepare_payload(); // no body, but still good practice

        // Send the request
        http::write(socket.next_layer(), request);
    }

    Response proxy_response;
    //... wait and process response
    {
        http::response_parser<http::empty_body> p;
        beast::flat_buffer                      buf;
        // Only headers expected
        http::read_header(socket.next_layer(), buf, p);

        proxy_response = std::move(p.get());

        assert(buf.size() == 0); // no excess data should be received
    }

    std::cout << proxy_response << "\n";
    if (proxy_response.result() == http::status::ok) {
        socket.set_verify_mode(ssl::verify_none);
        socket.handshake(ssl_socket::client);
        std::cout << "Handshake completed" << std::endl;
    } else {
        return 1; // TODO handle errors
    }

    {
        Request request(http::verb::get, "/", 10);
        request.set(http::field::host, in_server);
        http::write(socket, request);
    }

    {
        beast::flat_buffer buf;
        BodyResponse       http_res;
        http::read(socket, buf, http_res);
        std::cout << http_res;
    }
}

On my machine prints

HTTP/1.0 200 Connection established
Proxy-agent: tinyproxy/1.8.4


Handshake completed
HTTP/1.0 200 OK
Age: 214958
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Mon, 25 Oct 2021 23:11:37 GMT
Etag: "3147526947+gzip+ident"
Expires: Mon, 01 Nov 2021 23:11:37 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (bsa/EB23)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 1256
Connection: close

<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <style type="text/css">
    body {
        background-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
        
    }
    div {
        width: 600px;
        margin: 5em auto;
        padding: 2em;
        background-color: #fdfdff;
        border-radius: 0.5em;
        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
    }
    a:link, a:visited {
        color: #38488f;
        text-decoration: none;
    }
    @media (max-width: 700px) {
        div {
            margin: 0 auto;
            width: auto;
        }
    }
    </style>    
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is for use in illustrative examples in documents. You may use this
    domain in literature without prior coordination or asking for permission.</p>
    <p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>