Strtok removes first character in token C

Question

I have this little problem when trying to tokenize a string from a http request directed at my "home made" Http server.

Basicly I am using these lines of code to tokneize.

    token = strtok(bufptr, "\n");

    while(token != NULL){
        printf("%s \n", token);
        token = strtok(NULL, "\n");
    }

The problem is that for every token, the first character is removed from the tokenized string. How can I solve this?

I have tried copying the string, I have tried using strstr, but I have not yet succeded. I bet there is something pretty easy I am doing wrong.

Best regards.

Ok, but the exact content, can you print it out to the console with printf ? — George Nechifor, Aug 25 '14 at 13:18
@ThomasHolden I think that you need to use \r\n and not \n only. Have you considered it? — Alessandro Suglia, Aug 25 '14 at 14:03
This is the content of prtbuf. GET /test.php HTTP/1.1 Host: localhost:8080 Connection: keep-alive Cache-Control: max-age=0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like p4| — Thomas Holden, Aug 25 '14 at 19:04
Alessando: Thanks, it worked when I tried with \r\n. I think its wired though, because I read the UNIX does not use that delimiter for newline. — Thomas Holden, Aug 25 '14 at 19:11

score 3 · Accepted Answer · answered Aug 25 '14 at 15:38

You have a '\r' in the string. (@Alessandro Suglia)

 printf("%sx\n", "abc\r");  // x substituted for space

Prints:

xbc

The '\r' (return) move the print position to the beginning of the line. Then 'x' overwrites 'a'. This appears as the OP observed "the first character is removed from the tokenized string".

score 0 · Answer 2 · answered Aug 25 '14 at 13:15

0

This is working just fine:

char bufptr[] = "aaaa\nbbbb\ncccc\nddd";
char *token;
token = strtok(bufptr, "\n");

while(token != NULL){
    printf("%s \n", token);
    token = strtok(NULL, "\n");
}

output is:

aaaa 
bbbb 
cccc 
ddd

answered Aug 25 '14 at 13:15

Dr. Debasish Jana

6,980
4
30
69

score 0 · Answer 3 · answered Aug 25 '14 at 19:06

0

I got it working by using the \r\n as delimiter instead of only \n I did not think UNIX sent \r\n for newline, but seems that it does? Anyways, thanks.

answered Aug 25 '14 at 19:06

Thomas Holden

75
9

score 0 · Answer 4 · answered Aug 28 '14 at 21:14

For anyone else that is having this problem. The HTTP protocol uses the \r\n delimiters after the header lines, and a CRLN for seperating the body from the header. CRLN is \r\n\r\n as I understood it.

From my searching around the web I did not find alot easy readable information about the protocol. Of course, you do have the RFCs.

Thanks for the help everybody. Btw, I could never figure out how to accept an answer.

Strtok removes first character in token C

4 Answers4