3

I am trying to break string on delim "//". My string also contains "/" and StringTokenizer giving strange result, it also break string on "/".

  String  mStr = "abcd//aaa//32434//3/34343";
        StringTokenizer tok = new StringTokenizer(mStr, "//");
        while(tok.hasMoreTokens()){
            System.out.println(tok.nextToken());
        }

the result is

abcd
aaa
32434
3
34343

And the expected result is

    abcd
    aaa
    32434
    3/34343

Why this is happening and what is the solution of it? I do not want to replace "/" with other characters.

Rohit Jain
  • 209,639
  • 45
  • 409
  • 525
Talha
  • 699
  • 2
  • 12
  • 33

4 Answers4

9

StringTokenizer takes both the tokens as separate, and tokenizes on both of them. So, it is tokenizing on both // and /, and hence the result.


I would rather prefer String#split over StringTokenizer. It's easier to use, and has more options. It can take Regex as parameter, and returns an array of tokens which you can use later on: -

String  mStr = "abcd//aaa//32434//3/34343";

String[] arr = mStr.split("//");
System.out.println(Arrays.toString(arr));

Output : -

[abcd, aaa, 32434, 3/34343]
Rohit Jain
  • 209,639
  • 45
  • 409
  • 525
3

From http://docs.oracle.com/javase/1.4.2/docs/api/java/util/StringTokenizer.html:

An instance of StringTokenizer behaves in one of two ways, depending on whether it was created with the returnDelims flag having the value true or false:

  • If the flag is false, delimiter characters serve to separate tokens. A token is a maximal sequence of consecutive characters that are not delimiters.
  • If the flag is true, delimiter characters are themselves considered to be tokens. A token is thus either one delimiter character, or a maximal sequence of consecutive characters that are not delimiters.

StringTokenizer will always split on consecutive delimiter characters, not on the delimiter string itself. Thus, your split is "any number of slashes or slashes". // is two slashes or slashes; / is one slash or slash. Thus, your StringTokenizer will match both of the sequences as separators.

In other words, your StringTokenizer is equivalent to splitting on the following regular expression: [//]+, and not // as you thought.

Community
  • 1
  • 1
Amadan
  • 191,408
  • 23
  • 240
  • 301
2

StringTokenizer is a very simple class. If you want a tokenizer-like functionality, you can use Scanner, as it considers delimeter string as a regular expression.

    String mStr = "abcd//aaa//32434//3/34343";
    Scanner scanner = new Scanner(mStr);
    scanner.useDelimiter("//");
    while (scanner.hasNext()) {
        System.out.println(scanner.next());
    }
Denis Tulskiy
  • 19,012
  • 6
  • 50
  • 68
1

- split() method of String would do just fine, with // with delimiter.

public class Reg {

    public static void main(String[] args){

        String s = "abcd//aaa//32434//3/34343";
        String[] arr = s.split("//");

                for (String x : arr){

            System.out.println(x);
        }
       }
}
Kumar Vivek Mitra
  • 33,294
  • 6
  • 48
  • 75