WinInet - can't remove Content-Length from headers to manually chunk an upload

Question

MSDN states that WinInet does not support chunked upload ("Client code must perform the chunking."). To me that meant I could manually chunk the transfer. My intention was to add "Transfer-Encoding: chunked" via HttpAddRequestHeaders, remove Content-Length via HttpAddRequestHeaders HTTP_ADDREQ_FLAG_REPLACE and then In my InternetWriteFile loop, write the data in chunked encoded blocks. The problem is, I cannot seem to convince WinInet to not send the Content-Length. Even after removal, it ends up sending "Content-Length: 0" to the server (in addition to "Transfer-Encoding: chunked") which is confusing the server.

I have also tried setting the HSR_CHUNKED flag in HttpSendRequestEx.

Does anyone have an example of getting WinInet to skip sending the Content-Length?

I know WinHTTP claims to support chunked upload, but we have other dependencies on WinInet which is why I am looking to solve the problem there if possible.

Here is a code sample of what I have tried:

#include <windows.h>
#include <wininet.h>
#include <tchar.h>
#include <stdio.h>

bool http_putfile(HINTERNET hConnection, TCHAR* resource);

#define HOST  _T("www.website.com")
#define RESOURCE _T("/path/for/resource")

int _tmain(int argc, TCHAR* argv[])
{
    LPCTSTR lpszHostName = HOST;
    INTERNET_PORT nServerPort = INTERNET_DEFAULT_HTTP_PORT;
    DWORD dwService = INTERNET_SERVICE_HTTP;
    DWORD dwFlags = NULL;
    DWORD dwContext = 0;

    HINTERNET hSession = InternetOpen(
        argv[0],
        INTERNET_OPEN_TYPE_DIRECT, 
        NULL,
        NULL,
        NULL);

    if(hSession != NULL)
    {
        HINTERNET hConnection = InternetConnect(
            hSession,
            lpszHostName,
            nServerPort,
            NULL,
            NULL,
            dwService,
            dwFlags,
            dwContext);

        if(hConnection != NULL)
        {
            http_putfile(hConnection, RESOURCE);

            InternetCloseHandle(hConnection);
        }
        else
        {
            printf("InternetConnect failed: %d\n", GetLastError());
        }

        InternetCloseHandle(hSession);
    }
    else
    {
        printf("InternetOpen failed: %d\n", GetLastError());
    }

    return 0;
}

bool http_putfile(HINTERNET hConnection, TCHAR* resource)
{
    bool result = false;

    HINTERNET hRequest = HttpOpenRequest(
        hConnection,
        _T("PUT"),
        resource,
        NULL,
        NULL,
        NULL,
        INTERNET_FLAG_RELOAD | INTERNET_FLAG_EXISTING_CONNECT | INTERNET_FLAG_NO_CACHE_WRITE,
        0);

    if(hRequest != NULL)
    {
        HttpAddRequestHeaders(
            hRequest,
            _T("Transfer-Encoding: chunked\r\n"),
            -1L,
            HTTP_ADDREQ_FLAG_ADD | HTTP_ADDREQ_FLAG_REPLACE);

        // have tried:
        // Content-Length
        // Content-Length:
        // Content-Length\r\n
        // Content-Length:\r\n
        // all with/without HTTP_ADDREQ_FLAG_ADD.  Have even tried adding in a Content-Length
        // and then removing it.  All results show "Content-Length: 0" in the header on the wire.
        if(HttpAddRequestHeaders(
            hRequest,
            _T("Content-Length"),
            -1L,
            HTTP_ADDREQ_FLAG_ADD | HTTP_ADDREQ_FLAG_REPLACE) == FALSE)
        {
            DWORD err = GetLastError();
        }

        // have tried both 0 here for flags as documented on msdn
        // http://msdn.microsoft.com/en-us/library/windows/desktop/aa384318%28v=vs.85%29.aspx
        // as well as other combinations of INITIATE/CHUNKED
        if(HttpSendRequestEx(hRequest, NULL, NULL, HSR_INITIATE | HSR_CHUNKED /* 0 */, NULL))
        {
            DWORD wrote = 0;
            char* chunks = "5\r\nCHUNK0\r\n";

            if(InternetWriteFile(
                    hRequest,
                    chunks,
                    strlen(chunks),
                    &wrote) == FALSE)
            {
                printf("InternetWriteFile failed: %d\n", GetLastError());
            }

            HttpEndRequest(hRequest, NULL, 0, NULL);
        }
        else
        {
            printf("HttpSendRequestEx failed: %d\n", GetLastError);
        }

        InternetCloseHandle(hRequest);
    }
    else
    {
        printf("HttpOpenRequest failed: %d\n", GetLastError());
    }

    return result;
}

1. InternetOpen(), 2. InternetConnect(), 3. HttpOpenRequest(), 4. HttpAddRequestHeaders(), 5. HttpSendRequestEx(), 6. InternetWriteFile(), 7. HttpEndRequest(), 8. InternetCloseHandle() — ribram, Apr 09 '12 at 14:05

score 0 · Answer 1 · answered Apr 09 '12 at 20:36

0

Well, it looks like you are right. In fact, I don't see anything to indicate that WinINet supports chunked uploading; only WinHTTP does. Looking at the Wine source code (which tends to be quite accurate), the "Content-Length" header is ALWAYS appended to the request for any method other than "GET".

According to the HTTP specs, if a non-identity "Transfer-Encoding" header is present then the message is considered to be chunked. If a "Content-Length" header is also present then it MUST be ignored. If the spurious "Content-Length: 0" header is causing you problems then it could be argued that it is an issue with the server. I would give WinHTTP a try and see if it fixes the problem.

answered Apr 09 '12 at 20:36

Luke

11,211
2
27
38

You are correct, testing against a different HTTP server, it ignored the Content-Length and just accepted the chunked upload. RFC 2616 indicates that the client should never do this, but if it does, the server should behave as you stated. So it seems like both sides were wrong in my original case. Even though I can't use WinHTTP I did test it. I thought the docs implied it would directly do the chunking for you; it does not, you still have to chunk the data yourself but it does allow for deleting Content-Length from the headers. This is only supported on Vista/2k3 and above however. – ribram Apr 10 '12 at 22:31
Having arrived at a solution to this problem (albeit 18 months after the question was asked), I find something slightly irritating about your answer that it's not possible. – asveikau Oct 29 '13 at 15:28

score 0 · Answer 2 · answered Oct 17 '13 at 05:27

0

Great question, I was frustrated by this myself for a good number of hours.

Your instincts about HTTP_ADDREQ_FLAG_REPLACE turned out to be right... However, you have to call it pretty late. Specifically you need to register a status callback, and when your callback gets called with INTERNET_STATUS_CONNECTED_TO_SERVER, that is when you remove the header:

// Inside InternetStatusCallback
//
if (dwInternetStatus == INTERNET_STATUS_CONNECTED_TO_SERVER)
{
  HttpAddRequestHeaders(Handle, L"Content-Length", -1, HTTP_ADDREQ_FLAG_REPLACE);
}

answered Oct 17 '13 at 05:27

asveikau

39,039
2
53
68

I tried this on `Connection` header (don't ask me why), but it didn't work. – jwalker Dec 09 '14 at 20:50
@jwalker - Hm, for your case I wonder if the library itself is setting that header later? There might be another `dwInternetStatus` that you can look for. – asveikau Dec 09 '14 at 22:03

WinInet - can't remove Content-Length from headers to manually chunk an upload

2 Answers2