unable to understand Output of the following simple program

Question

/* A Java program to illustrate working of StringTokenizer
class:*/
import java.util.*;
public class NewClass
{
    public static void main(String args[])
    {


        System.out.println("Using Constructor 3 - ");
        StringTokenizer st3 = 
            new StringTokenizer("JAVA : Code : String", " :", true);
        while (st3.hasMoreTokens())
            System.out.println(st3.nextToken());
    }
}

Why is the output of the above program is as follows:

Using Constructor 3 - 
JAVA

:

Code

:

String

My question is why it has extra newline characters(i.e. it has blank lines after "Java", ":", "Code" etc)

Answer 1

It will become a bit clearer if we make the printing a bit more verbose:

while (st3.hasMoreTokens())
  System.out.printf("token: '%s'\n", st3.nextToken());

The output will be:

token: 'JAVA'
token: ' '
token: ':'
token: ' '
token: 'Code'
token: ' '
token: ':'
token: ' '
token: 'String'

As you see, and : are tokens. It's because every character in the specified delimiter string : are used as delimiters. And you passed true as the 3rd parameter to the constructor of StringTokenizer, and that makes it return the delimiters as tokens.

This is explained in the JavaDoc of StringTokenizer.html#StringTokenizer.

Answer 2

Because the true argument in your constructor means return delimiters as separate tokens. So, in addition to the words, you get one token for each space (which appears as a blank line), as well as the semicolon.

Answer 3

You are defining - StringTokenizer . It will split your input by colons and spaces due to " :"; you have defined two tokens, space and colon. Next you are looping through the tokens and doing println i.e. printing a new line per token.