C++ WinHttp get response header and body

Question

I'm currently trying to write a class to make sending simple requests easier for me.
In the end I'd like it to be usable somewhat like this:

int _tmain(int argc, _TCHAR* argv[])
{
    HttpRequest Request(L"Example UserAgent/1.0",L"",L"");
    Request.SendRequest(L"google.com",L"GET",NULL);
    if (Request.responseHeader)
        printf("%s",Request.responseHeader);
    if (Request.responseBody)
        printf("%s",Request.responseBody);
    getchar();
    return 0;
}

But for now it doesn't work at all. I have no idea how I could get the response header and I'm failing writing the response header to a public member of my class.
Yeah I'm really bad at C++ especially when it's about the winapi.
Well I hope you can help me out.
Here is my code so far:

#include "stdafx.h"
#include <windows.h>
#include <winhttp.h>
#pragma comment(lib, "winhttp.lib")

class HttpRequest {
  private:
    DWORD dwSize;
    DWORD dwDownloaded;
    LPSTR pszOutBuffer;
    BOOL  bResults;
    HINTERNET hSession;
    HINTERNET hConnect;
    HINTERNET hRequest;
    LPCWSTR _userAgent;
    //LPCWSTR _proxyIp;
    //LPCWSTR _proxyPort;
    size_t bodySize;
  public:
    HttpRequest(LPCWSTR, LPCWSTR, LPCWSTR);
    void SendRequest(LPCWSTR, LPCWSTR, LPVOID);
    LPSTR responseHeader[1000000];
    LPSTR responseBody[1000000];
};

HttpRequest::HttpRequest(LPCWSTR userAgent, LPCWSTR proxyIp, LPCWSTR proxyPort) {
    _userAgent = userAgent;
    //_proxyIp = proxyIp;
    //_proxyPort = proxyPort;
    hSession = WinHttpOpen( userAgent, WINHTTP_ACCESS_TYPE_DEFAULT_PROXY, WINHTTP_NO_PROXY_NAME, WINHTTP_NO_PROXY_BYPASS, 0 );
}

void HttpRequest::SendRequest(LPCWSTR url, LPCWSTR method, LPVOID body) {
    bodySize = 0;
    if (hSession)
        hConnect = WinHttpConnect( hSession, url, INTERNET_DEFAULT_HTTPS_PORT, 0 );
    else
        printf("session handle failed\n");
    if (hConnect)
        hRequest = WinHttpOpenRequest( hConnect, method, NULL, NULL, WINHTTP_NO_REFERER, WINHTTP_DEFAULT_ACCEPT_TYPES, WINHTTP_FLAG_SECURE );
    else
        printf("connect handle failed\n");
    if (hRequest)
        bResults = WinHttpSendRequest( hRequest, WINHTTP_NO_ADDITIONAL_HEADERS, 0, body, 0, 0, 0 );
    else
        printf("request handle failed\n");

    if( bResults )
        bResults = WinHttpReceiveResponse( hRequest, NULL );
    if( bResults )
    {
        do 
        {
            // Check for available data.
            dwSize = 0;
            if( !WinHttpQueryDataAvailable( hRequest, &dwSize ) )
                printf( "Error %u in WinHttpQueryDataAvailable.\n", GetLastError( ) );

            // Allocate space for the buffer.
            pszOutBuffer = new char[dwSize+1];
            if( !pszOutBuffer )
            {
                printf( "Out of memory\n" );
                dwSize=0;
            }
            else
            {
                // Read the data.
                ZeroMemory( pszOutBuffer, dwSize+1 );

                if( !WinHttpReadData( hRequest, (LPVOID)pszOutBuffer, dwSize, &dwDownloaded ) )
                    printf( "Error %u in WinHttpReadData.\n", GetLastError( ) );
                else { 
                    //printf( "%s", pszOutBuffer );
                    responseBody[bodySize++] = pszOutBuffer; 
                }
                // Free the memory allocated to the buffer.
                delete [] pszOutBuffer;
            }
        } while( dwSize > 0 );
    }

    // Report any errors.
    if( !bResults )
        printf( "Error %d has occurred.\n", GetLastError( ) );

    // Close any open handles.
    if( hRequest ) WinHttpCloseHandle( hRequest );
    if( hConnect ) WinHttpCloseHandle( hConnect );
    if( hSession ) WinHttpCloseHandle( hSession );
}

Remy Lebeau · Answer 1 · 2014-04-15T03:06:14.990

Use WinHttpQueryHeaders() to access the response headers. Use the WINHTTP_QUERY_RAW_HEADERS(_CRLF) flag to specify that you want to retrieve all of the available headers.

You also need to change your class to dynamically allocate its responseHeader and responseBody members. You are wasting a lot of memory, as well as limiting the response size you can handle, by using static arrays.

Try this:

#include "stdafx.h"
#include <windows.h>
#include <winhttp.h>
#include <string>
#include <vector>

#pragma comment(lib, "winhttp.lib")

class HttpRequest
{
private:
    std::wstring _userAgent;
    //std::wstring _proxyIp;
    //std::wstring _proxyPort;
public:
    HttpRequest(const std::wstring&, const std::wstring&, const std::wstring&);
    bool SendRequest(const std::wstring&, const std::wstring&, void*, DWORD);
    std::wstring responseHeader;
    std::vector<BYTE> responseBody;
};

HttpRequest::HttpRequest(const std::wstring &userAgent, const std::wstring &proxyIp, const std::wstring &proxyPort) :
    _userAgent(userAgent)
    //,_proxyIp(proxyIp)
    //,_proxyPort(proxyPort)
{
}

bool HttpRequest::SendRequest(const std::wstring &url, const std::wstring &method, void *body, DWORD bodySize)
{
    DWORD dwSize;
    DWORD dwDownloaded;
    DWORD headerSize = 0;
    BOOL  bResults = FALSE;
    HINTERNET hSession;
    HINTERNET hConnect;
    HINTERNET hRequest;

    responseHeader.resize(0);
    responseBody.resize(0);

    hSession = WinHttpOpen( _userAgent.c_str(), WINHTTP_ACCESS_TYPE_DEFAULT_PROXY, WINHTTP_NO_PROXY_NAME, WINHTTP_NO_PROXY_BYPASS, 0 );
    if (hSession)
        hConnect = WinHttpConnect( hSession, url.c_str(), INTERNET_DEFAULT_HTTPS_PORT, 0 );
    else
        printf("session handle failed\n");

    if (hConnect)
        hRequest = WinHttpOpenRequest( hConnect, method.c_str(), NULL, NULL, WINHTTP_NO_REFERER, WINHTTP_DEFAULT_ACCEPT_TYPES, WINHTTP_FLAG_SECURE );
    else
        printf("connect handle failed\n");

    if (hRequest)
        bResults = WinHttpSendRequest( hRequest, WINHTTP_NO_ADDITIONAL_HEADERS, 0, body, bodySize, 0, 0 );
    else
        printf("request handle failed\n");

    if (bResults)
        bResults = WinHttpReceiveResponse( hRequest, NULL );
    if (bResults)
    {
        bResults = WinHttpQueryHeaders(hRequest, WINHTTP_QUERY_RAW_HEADERS_CRLF, NULL, WINHTTP_NO_OUTPUT_BUFFER, &headerSize, WINHTTP_NO_HEADER_INDEX);
        if ((!bResults) && (GetLastError() == ERROR_INSUFFICIENT_BUFFER))
        {
            responseHeader.resize(headerSize / sizeof(wchar_t));
            if (responseHeader.empty())
            {
                bResults = TRUE;
            }
            else
            {
                bResults = WinHttpQueryHeaders(hRequest, WINHTTP_QUERY_RAW_HEADERS_CRLF, NULL, &responseHeader[0], &headerSize, WINHTTP_NO_HEADER_INDEX);
                if( !bResults ) headerSize = 0;
                responseHeader.resize(headerSize / sizeof(wchar_t));
            }
        }
    }
    if (bResults)
    {
        do
        {
            // Check for available data.
            dwSize = 0;
            bResults = WinHttpQueryDataAvailable( hRequest, &dwSize );
            if (!bResults)
            {
                printf( "Error %u in WinHttpQueryDataAvailable.\n", GetLastError( ) );
                break;
            }

            if (dwSize == 0)
                break;

            do
            {
                // Allocate space for the buffer.
                DWORD dwOffset = responseBody.size();
                responseBody.resize(dwOffset+dwSize);

                // Read the data.
                bResults = WinHttpReadData( hRequest, &responseBody[dwOffset], dwSize, &dwDownloaded );
                if (!bResults)
                {
                    printf( "Error %u in WinHttpReadData.\n", GetLastError( ) );
                    dwDownloaded = 0;
                }

                responseBody.resize(dwOffset+dwDownloaded);

                if (dwDownloaded == 0)
                    break;

                dwSize -= dwDownloaded;
            }
            while (dwSize > 0);
        }
        while (true);
    }

    // Report any errors.
    if (!bResults)
        printf( "Error %d has occurred.\n", GetLastError( ) );

    // Close any open handles.
    if( hRequest ) WinHttpCloseHandle( hRequest );
    if( hConnect ) WinHttpCloseHandle( hConnect );
    if( hSession ) WinHttpCloseHandle( hSession );

    return bResults;
}

int _tmain(int argc, _TCHAR* argv[])
{
    HttpRequest Request(L"Example UserAgent/1.0",L"",L"");
    if (Request.SendRequest(L"google.com",L"GET",NULL,0))
    {
        printf("%ls",Request.responseHeader.c_str());
        if (!Request.responseBody.empty())
            printf("%*s",Request.responseBody.size(),(char*)&Request.responseBody[0]);
    }
    getchar();
    return 0;
}

Thank you. There seems to be something wrong with the vectors though. I've never used vectors and I'm not sure if I understand them. I get the following error: Debug Assertion Failed! | File: c:\program files (x86)\microsoft visual studio 10.0\vc\include\vector | Line: 932 | Expression: vector subscript out of range ... I'm also wondering how I would get the byte size of the request body and if it would be possible to calculate it in the function. Here is the modified version: http://pastebin.com/ye8JUm4A the request and the response are sent correctly, the app just fails with the vectors... — Forivin, Apr 15 '14 at 02:42
Works fine when I try it, no out of range errors. However, I have tweaked the code for you, just in case. When `SendRequest()` returns true, you can retreive the body size via `responseBody.size()`. It is not always possible to get the body size ahead of time, for instance when HTTP's `Transfer-Encoding: chunked` header is used. So you just keep reading until the end of the response is reached. — Remy Lebeau, Apr 15 '14 at 03:06
I don't get it, why would only I get this error. How is that possible? What could I be doing wrong? — Forivin, Apr 15 '14 at 04:29
Well, I've never used vectors before. I have absolutely no idea how they work. [...] if (size() <= _Pos) { // report error [...] I guess that means that responseBody.resize(dwOffset+dwSize); didn't make the vector large enough for the data that was being read into it. But from what I can tell it looks right. Maybe I just need to change a setting in my vc++? I mean since the code per se works for you, it should be correct. — Forivin, Apr 15 '14 at 05:17
A vector is just a dynamic array, like a wrapper around `new[]` and `delete[]`. If you are going to code in C++ then you should get familiar with C++'s clases. If you are more comfortable with `new`/`delete` then switch, but you are likely still going to have problems if you are not doing the right things. There is no `_Pos` in the code I gave you, so what exactly are you validating? I tested the code that I gave you. — Remy Lebeau, Apr 15 '14 at 15:49
The debugger pointed me to the code with the _Pos. It's part of the vector lib. I had to change lots of things in the code you gave me in the first place btw. There were some issues with the arguments of the constructor, with {, } and :, with responseHeader being called responseHeaders sometimes. I also don't think that this line is valid: _userAgent(userAgent) ... I just changed it to _userAgent = userAgent; ... Well I corrected these and added some other small things like accepting proxies and a request header. The added features work just fine. It just only when it gets to the vectors. — Forivin, Apr 15 '14 at 16:17
The code I gave you compiles and runs just fine in my compiler. Yes, my original code had some typos, but I fixed those. Did you not try the new code yet? And yes, `_userAgent(userAgent)` is valid code, it is in the constructor's **initialization list** (see the `:` before the list?). — Remy Lebeau, Apr 15 '14 at 18:46
You edited the code? Well, that explains a lot. I'll check it out later. — Forivin, Apr 16 '14 at 01:46

C++ WinHttp get response header and body

1 Answers1