0

I tried the code blow for downloading file from a Rest API with boost asio, the consequence was it look like just had a small portions of file in the body of response , I had no idea what wrong I have done,Boost is latest version within VS2013 on a Win7 machine

    boost::asio::io_service io_service;

    // Get a list of endpoints corresponding to the server name.
    boost::asio::ip::tcp::resolver resolver(io_service);
    boost::asio::ip::tcp::resolver::query query(host, port);
    boost::asio::ip::tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);

    // Try each endpoint until we successfully establish a connection.
    boost::asio::ip::tcp::socket socket(io_service);
    boost::asio::connect(socket, endpoint_iterator);

    // Form the request. We specify the "Connection: close" header so that the
    // server will close the socket after transmitting the response. This will
    // allow us to treat all data up until the EOF as the content.
    boost::asio::streambuf request;
    std::ostream request_stream(&request);
    request_stream << "POST " << target << " HTTP/1.1\r\n";
    request_stream << "Host: " << host << "\r\n";
    request_stream << "Accept: " << acceptwstr << "\r\n";
    request_stream << "Content-Type: " << ContentTypewstr << "\r\n";
    request_stream << "Content-Length: " << jsondata.length() << "\r\n";
    request_stream << "x-tif-paasid: " << accessKey << "\r\n";
    request_stream << "x-tif-timestamp: " << msecondstr << "\r\n";
    request_stream << "x-tif-nonce: " << uuidstr << "\r\n";
    request_stream << "x-tif-signature: " << signature << "\r\n";
    request_stream << "Connection: Close\r\n\r\n";
    request_stream << jsondata << "\r\n";

    // Send the request.
    boost::asio::write(socket, request);

    // Read the response status line. The response streambuf will automatically
    // grow to accommodate the entire line. The growth may be limited by passing
    // a maximum size to the streambuf constructor.
    boost::asio::streambuf response;
    boost::asio::read_until(socket, response, "\r\n");

    // Check that response is OK.
    std::istream response_stream(&response);
    std::string http_version;
    response_stream >> http_version;
    unsigned int status_code;
    response_stream >> status_code;
    std::string status_message;
    std::getline(response_stream, status_message);
    if (!response_stream || http_version.substr(0, 5) != "HTTP/")
    {
        //std::cout << "Invalid response\n";
        return 9002;
         
    }
    if (status_code != 200)
    {
        //std::cout << "Response returned with status code " << status_code << "\n";
        return 9003;
    }

    // Read the response headers, which are terminated by a blank line.
    boost::asio::read_until(socket, response, "\r\n\r\n");

    // Process the response headers.
    std::string header;
    std::string fullHeader = "";
    while (std::getline(response_stream, header) && header != "\r")
        fullHeader.append(header).append("\n");

    // Write whatever content we already have to output.
    std::string fullResponse = "";
    if (response.size() > 0)
    {
        std::stringstream ss;
        ss << &response;
        fullResponse = ss.str();
     
    
    }

    // Read until EOF, writing data to output as we go.
    std::string fullSth = "";
    boost::system::error_code error;
    while (boost::asio::read(socket, response,
        boost::asio::transfer_at_least(1), error))
    {
        std::ostringstream ss;
        ss << &response;
        fullSth = ss.str();
    }
    if (error != boost::asio::error::eof)
        throw boost::system::system_error(error);
}
catch (std::exception& e)
{
    write_text_to_log_file(e.what());
    return 9001;
}

the value of fullHeader was

 x-proxy-by: Tif-APIGate
 Set-Cookie: XSRF-TOKEN=ebcf70d2-8b00-46b9-a8f9-6dcbbc535ee9; Path=/
 Content-Disposition: attachment; filename="202012303333629003180526491.txt.zip"
 X-Content-Type-Options: nosniff
 X-XSS-Protection: 1; mode=block
 Cache-Control: no-cache, no-store, max-age=0, must-revalidate
 Pragma: no-cache
 Expires: 0
 Content-Type: application/octet-stream;charset=UTF-8
 Transfer-Encoding: chunked
 Date: Tue, 23 Mar 2021 02:34:38 GMT
 connection: Close
 x-tif-nonce: kmlen6de-122075580

the value of fullResponse was

 2a4
 PK
Ken
  • 47
  • 6

1 Answers1

0

If you intend to read until EOF, is seems that instead of transfer_at_least(1) you should say transfer_all().

Also, if you do write the loop, you should use the results, not overwrite it. So, e.g. instead of

        fullSth = ss.str();

I'd expect something more like

        fullSth += ss.str();
sehe
  • 374,641
  • 47
  • 450
  • 633
  • thanks for your time, I found the problem was Chunked Transfer, it's way more compliacted than normal transfer due to it didn't metion the length in the response header, I'm trying something else which like https://stackoverflow.com/questions/40780782/slow-http-client-based-on-boostasio-chunked-transfer , – Ken Mar 23 '21 at 08:45
  • I really apperciate your helps – Ken Mar 23 '21 at 08:46
  • @Ken Can I kindly suggest doing it MUCH simpler: https://stackoverflow.com/questions/66730771/how-to-fix-boostasio-client-http-request-error/66738336#answer-66738336:~:text=If%20you%20want%20to%20be%20really%20solid%2C%20use%20Beast%20to%20receive%20a%20HTTP%2F1.1%20response%20(which%20can%20even%20be%20chunked). There's really no need why anyone is still copy pasting bad HTTP implementations as if it's 1992 (The Asio docs should make it clear that the samples are for expositional puposes only, and that Boost Beast exists) – sehe Mar 23 '21 at 09:06
  • Hi sorry for getting back you late, these days I Googled how to handle the Chunked encode datas,thanks for the inspiration, I have an idea how to deal with it and now I'm trying to parse it , thank you so much,really apperciated it – – Ken Mar 25 '21 at 07:28