C parsing HTTP Chunked transfer encoding response

Question

I am developing a client that needs to parse Chunked-type HTTP transfers. I've beat my head against the wall trying to figure out the error with the following, and would appreciate it if someone might be able to catch my error a bit quicker. To sum up the issue: it seems as though, the client does not receive ALL of the chunk, thereby screwing up the rest of the process. Thanks in advance!

 while(cflag){
    pfile_chunk = malloc(CHUNK_SIZE+1);
    memset(pfile_chunk, 0, CHUNK_SIZE);
    cPtr = pfile_chunk;
    cPtr2 = NULL;
    k=0;
    while(*(cPtr-1) != '\n'){
        k++;
        recv(sock, cPtr, 1, 0);
        cPtr = pfile_chunk+k;
    }
    cPtr2 = strchr(pfile_chunk, '\r');
    *cPtr2 = '\0';
    sscanf(pfile_chunk, "%x", &l);
    if(l == 0)
        break;
    printf("\nServer wants to deliver %ld bytes.\n", l);
    pfile_chunk = realloc(pfile_chunk, l+1);
    memset(pfile_chunk, 0, l);
    recv(sock, pfile_chunk, l, 0);
    fputs(pfile_chunk, f);
    printf("GOT THIS, SIZE %ld:\n%s\n", strlen(pfile_chunk), pfile_chunk);
    //get next \r\n bytes.
    recv(sock, NULL, 2, 0);
}

I apologize in advance for the sloppy code, it was more or less a result of me trying many different things. — E Klonowski, Jul 23 '12 at 22:05
Would you mind posting your final, cleaned-up code? I am struggling with this task as well. Not sure how to reconcile what's read in from the buffer with the actual HTTP chunks. Do you store the entire HTTP response before parsing the chunks? — CFL_Jeff, Aug 24 '12 at 09:02

jxh · Accepted Answer · 2012-07-23T22:32:48.343

5

At the very least, you should check the return value of recv to see if you are getting the number of bytes you are expecting to get.

A short read is definitely possible on the network, since the system call will return whatever is available in the socket receive buffer at the time you make the call.

Implement a loop until you have read in your entire chunk, or pass the MSG_WAITALL flag to recv in the last parameter. However, you still need to check for an error from recv.

ssize_t r = recv(sock, pfile_chunk, l, MSG_WAITALL);
if (r < l) {
    /* check for errors ... */
} else {
    /* got the data */
}

edited Jul 23 '12 at 22:32

answered Jul 23 '12 at 22:27

jxh

69,070
8
110
193

Thankyou sir, I didn't even consider send/recv flags. The error checking wasn't present as I'd been in such a rush moving sloppy code around. Appreciate it! Somebody +1 this guy, I don't have the required reputation. – E Klonowski Jul 24 '12 at 00:51

score 2 · Answer 2 · edited Oct 11 '12 at 18:53

It looks as though your very first dereference for the check in your while loop will access before the beginning of your array, which is likely not to be desired behavior. Hopefully, that memory location usually won't contain \n. That could mess up your read. I expect it probably contains some information to do with your malloc, which is unlikely to be \n, so you might never see a problem from that.

Also, hopefully you can trust the other end of the socket not to send more than CHUNK_SIZE+1 before they give you a \n. Otherwise, it could seg-fault out. Normally, though, I would expect a sender to just send 10 or fewer ASCII numeric characters and a CRLF for a chunk header anyways, but they could theoretically send a bunch of long chunk extension header fields with it.

Apart from that, there's just the more important issue already found by user315052 that you should either tell the recv method to wait for all the data you requested, or check how much data it actually read.

C parsing HTTP Chunked transfer encoding response

2 Answers2