strtok deletes my string except the first token

Question

i have linked list with songs details and want to compare two song names... the additional task is to compare them by having more similar words than different words...i use strtok to get it word by word and then check if the current word is contained in the other name...but when i do it strtok clear my string except the first word...how can i escape from that?

   while (next_token)
            {
                currentCounter++;
                if (searchword(temp->author, next_token)==1)
                {
                    counter++;
                }
                next_token = strtok(NULL, " ");

            }

Did you read the documentation for what `strtok()` *actually does*? — EOF, May 05 '17 at 21:08
`strtok()` works by overwriting the delimiters with null bytes, so that each token becomes a separate string. If you need the original string, make a copy of it before using `strtok()`. — Barmar, May 05 '17 at 21:08
no no.... for example temp->author is "bon jovi"....on the first itteration next_token gets"bon" but on the second deletes everything from temp->author except "bon" and cant go to next token — Айрола Ходжов, May 05 '17 at 21:10
Are you saying you're using `temp->author` in both the `if` and the initial call to `strtok` ? — Phil M, May 05 '17 at 21:15
temp->author is a help list...its some sort of a copy... in strtok i use current->author — Айрола Ходжов, May 05 '17 at 21:22
Where is your first strtok() statement? You need something like `next_token = strtok( string, " ");` That should be there before using `next_token = strtok(NULL, " ");` — Nguai al, May 05 '17 at 21:29
yes i have all needed for strtok...i read about two hours about it and i did a simple program to see how it works and it worked...but then when i started using strtok in my project it gets the first token from the string and then removes everything from it except the taken token — Айрола Ходжов, May 05 '17 at 21:31
What is so difficult about just providing what those who are trying to help you asked for? Show the original input string, and the very first call to `strtok(.);`, exactly as you have them in the code that is failing you. — ryyker, May 05 '17 at 22:07
What you're explaining sounds exactly like the symptoms Barmar describes. He is right, you need to make a copy of the original before you call strtok because strtok WILL modify your variable. — nugenjs, May 05 '17 at 22:19

score 2 · Accepted Answer · answered May 05 '17 at 22:08

strtok modifies the input array by overwriting the delimiters with 0s; it's not that everything after the first token has been removed or cleared, it's just that you have a string terminator following the first token, so nothing after it gets printed.

For example:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>

int main( void )
{
  char str[] = "This is a test";
  printf( "before strtok: " );
  for ( size_t i = 0; i < sizeof str; i++ )
  {
    if ( isprint( str[i] ) )
      putchar( str[i] );
    else
      putchar( '.' );
  }
  putchar ( '\n' );

  char *token = strtok( str, " " );
  while ( token )
  {
    printf( "token = %8s, str = ", token );
    for ( size_t i = 0; i < sizeof str; i++ )
    {
      if ( isprint( str[i] ) )
        putchar( str[i] );
      else
        putchar( '.' );
    }
    putchar( '\n' );
    token = strtok( NULL, " " );
  }

  printf( "after all strtok: " );
  for ( size_t i = 0; i < sizeof str; i++ )
  {
    if ( isprint( str[i] ) )
      putchar( str[i] );
    else
      putchar( '.' );
  }
  putchar( '\n' );

  return 0;
}

Here's the output (a . represents a 0 in this case):

before strtok: This is a test.
token =     This, str = This.is a test.
token =       is, str = This.is.a test.
token =        a, str = This.is.a.test.
token =     test, str = This.is.a.test.
after all strtok: This.is.a.test.

After the strtok calls, all the other tokens are still there, but because we've overwritten the blank spaces with 0s, all the regular text processing routines (strcpy, strcat, printf, puts, etc.) only see the first token.

If you want to preserve the contents of the original string, you will have to copy them into another buffer and perform the strtok calls on that buffer.

Thank you very much John Bode! – Айрола Ходжов May 05 '17 at 22:14 — Айрола Ходжов, May 05 '17 at 22:14

strtok deletes my string except the first token

1 Answers1