3

Given a string in the form of:

String myStr = "5.1\t3.5\t1.4\t0.2\t0.0";

If I call:

StringTokenizer token = new StringTokenizer(myStr, "\\s+");
String firstElement = token.nextToken();

firstElement then equals the whole string. In contrast, if I call:

StringTokenizer token2 = new StringTokenizer(myStr);
String firstElement = token2.nextToken();

firstElement equals "5.1". Similarly if I use String split as below:

String[] splitArray = myStr.split("\\s+")
String firstElement = splitArray[0];

then, firstElement is "5.1".

I understand StringTokenizer is discouraged for use and is a classified as a "legacy class". My intent here is to understand why the same delimiter works differently between split and StringTokenizer. I would have expected the first example to work like the latter two, but for some reason, it is skipping the tabs. Any guidance on what I am missing would be much appreciated.

Note I am running 1.7.0_19 on OSX in Eclipse, but I would not expect those variables to have an effect here.

ZaydH
  • 658
  • 6
  • 22

2 Answers2

4

StringTokenizer doesn't use a regular expression as the delimiters. The parameter is a string containing a list of delimiter characters.

The constructor StringTokenizer(String) is same as StringTokenizer(String, "\t\n\f\r") hence it works for your string.

Dakshinamurthy Karra
  • 5,353
  • 1
  • 17
  • 28
  • The use of StringTokenizer with "\\s+" is based off this Hadoop Tutorial from MapR. I assume then this is in error. https://www.mapr.com/blog/how-write-mapreduce-program#.VfZblvlVhBc – ZaydH Sep 14 '15 at 05:32
1

StringTokenizer: it uses delimiters as string which may contain list of delimiter characters not as regex

enter image description here

Split: it uses delimiters as regex

enter image description here

Rustam
  • 6,485
  • 1
  • 25
  • 25
  • For those like me who were not too savvy on regular expressions, I found this helpful. http://www.vogella.com/tutorials/JavaRegularExpressions/article.html – ZaydH Sep 14 '15 at 08:22