Why does strtok() append a space to the last token

Question

I am working on creating a simple version of Minix. I am using fgets() to grab user input. I am then using strtok() to split the string up with the delimiter " ". The problem is when I call strtok(NULL, " "), my token stored appends a space to the last char. So if I pass "minimount imagefile.img", my program will grab minimount and store it in variable cmd, then it will grab "imagefile.img " and place it in variable flag. Notice the space at the end of the flag variable is added after the token method.

Is there a way for me to grab just the string without a space at the end after token is called. Or is there a way to manipulate the string to remove the appended space?

printf("Minix: ");

fgets(cmd, BUFFLIM, stdin);

//parses string using delimiter " "

char *token = strtok(cmd, " ");

//assigns flag to what is after delimiter 

char *f = strtok(NULL, " ");

//printf("cmd:%s\nf:%s\n", cmd, f);

printf("cmd:%s\nf:%s", cmd, f);

Output:

cmd:"minimount"
f:"imagefile.img "

It is probably not really a space, but the newline, which `fgets()` adds to the buffer. You have to remove this newline manually. — Ctx, Oct 17 '19 at 14:24
This looks like it may be a bug. [`strtok()` is supposed to terminate that second token](https://port70.net/~nsz/c/c11/n1570.html#7.24.5.8p4). Please provide a *complete* example — Andrew Henle, Oct 17 '19 at 14:24
Your quoted output with `"` around the `flag` variable doesn't match the code — which has `f` instead of `flag` and doesn't print any double quotes. That leaves us wondering what goes with what — you need to make sure your code matches your question. — Jonathan Leffler, Oct 17 '19 at 14:43

Vlad from Moscow · Answer 1 · 2019-10-17T14:55:29.337

The standard function fgets can append the new line character '\n' to the entered string provided that there is enough space in the corresponding character array.

So use

char *f = strtok(NULL, " \n");

instead of

char *f = strtok(NULL, " ");

From the C Standard (7.21.7.2 The fgets function)

2 The fgets function reads at most one less than the number of characters specified by n from the stream pointed to by stream into the array pointed to by s. No additional characters are read after a new-line character (which is retained) or after end-of-file. A null character is written immediately after the last character read into the array.

Another approach is initially to remove the character from the entered string as for example

#include <string.h>
#include <stdio.h>

//...

fgets(cmd, BUFFLIM, stdin);
cmd[ strcspn( cmd, "\n" ) ] = '\0';

As for your code snippet then it seems you have the following result as it is shown in the demonstrative program.

#include <stdio.h>
#include <string.h>

int main(void) 
{
    char s[100];

    fgets( s, sizeof( s ), stdin );

    char *cmd = strtok( s, " " );

    char *f = strtok( NULL, " " );

    printf( "cmd:\"%s\"\nf:\"%s\"", cmd, f );

    return 0;
}

The program output is

cmd:"minimount"
f:"imagefile.img
"

Thank you for solving the problem. Issue was fgets() appends the newline char so using cmd[ strcspn( cmd, "\n" ) ] = '\0'; worked! — kCorcoran, Oct 17 '19 at 15:15

Why does strtok() append a space to the last token

1 Answers1