InternetReadFile reads only 10kb

Question

i'm trying to download file from http server using WinINet library calls. It works perfectly fine on my local web server. but when i'm trying download something from the internet, InternetReadFile reads only ~10kb of any file (text or binary). TRANSFER_SIZE = 4096 in the example below, thus there are two reads by 4kb and one by 2kb. Every next InternetReadFile returns true and 0 bytes read.

    hInternet = InternetOpen(L"Agent", INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0);
    DWORD options = INTERNET_FLAG_NEED_FILE|INTERNET_FLAG_HYPERLINK|INTERNET_FLAG_RESYNCHRONIZE|INTERNET_FLAG_RELOAD;
    HINTERNET hSession = InternetOpenUrl(hInternet, URL, NULL, NULL, options, 0);

    hFile = CreateFile(...);
    ...
    do {
         DWORD dwWriteSize, dwNumWritten;
         BOOL bRead = InternetReadFile(hSession, pBuf, TRANSFER_SIZE, &dwReadSizeOut);
         dwWriteSize = dwReadSizeOut;

         if (bRead && dwReadSizeOut > 0) {
             dwTotalReadSize += dwReadSizeOut;
             WriteFile(hFile, pBuf, dwWriteSize, &dwNumWritten, NULL); 
             // File write error
             if (dwWriteSize != dwNumWritten) {
                 CloseHandle(hFile);                    
                 return false;
             }
         }
         else {
             if (!bRead)
             {
                 // Error
                 CloseHandle(hFile);    
                 return false;
             }          
             break;
         }
     } while(1);

How can i download the entire file using WinINet library?

Did you run a packet sniffer (NetMon, wiresark, etc...) to detect if the web server delivering the URL is actually sending you the bytes you expect? — selbie, Jul 10 '11 at 07:30
InternetReadFile should behave on the Internet the same as it did on your local webserver. Something has changed somewhere; if it's not your code, then expand your troubleshooting. Double-check the response from the webserver is what you expect and no other hardware or software in the middle is causing the behavior. See this question for a great example of a similar problem with a surprising result: http://stackoverflow.com/questions/5771636/internetopenurl-only-returns-after-entire-http-response-is-downloaded/5772571 — J.J., Jul 15 '11 at 16:46
yes, checking the response was the solution. response was http 404 error - other guy from the project moved file to different directory =( that was so stupid! — polotenchiko, Jul 18 '11 at 09:18
@polotenchiko - make sure to answer (and accept) your own question so that we can find the answer in the future. — codekaizen, Jul 18 '11 at 09:20

score 2 · Answer 1 · answered Jul 15 '11 at 15:39

2

Try setting INTERNET_FLAG_KEEP_CONNECTION on InternetOpenURl.

You should also be doing at least HttpQueryInfo(HTTP_QUERY_STATUS_CODE | HTTP_QUERY_FLAG_NUMBER) on the handle after opening.

I would suggest looking at some existing wrapper C++ classes to these things, http://www.google.com/codesearch#search/&q=INTERNET_FLAG_KEEP_CONNECTION%20lang:c%2B%2B&type=cs

answered Jul 15 '11 at 15:39

kert

2,161
21
22

thanks! looks like if i didn't write a bunch of code by myself, i wouldn't spend so much time looking for errors there. also i was advised to use libcurl next time - looks fine and the licenses are ok. – polotenchiko Jul 18 '11 at 09:21
Thank you so much for mentioning INTERNET_FLAG_KEEP_CONNECTION – denis Feb 14 '17 at 23:30

score 0 · Accepted Answer · answered Jul 19 '11 at 14:50

0

I read the response as text and it was the "404 error" respond from the web server - file was missing. So it's useful to read the responses ;)

And libcurl looks like nice replacement of the WinINet library - easier to start with, a lot of options.

answered Jul 19 '11 at 14:50

polotenchiko

91
2
9

InternetReadFile reads only 10kb

2 Answers2