Obtain number of token in a string with java Scanner

Question

Is there a way to obtain the number of token in a string obtained by Method Scanner in Java?

I mean, i can use s = sc.nextLine() to obtain an input line as a string. But on this string I cannot use lenght() method cause it gives me the total number of characters (I think).

Are existing any standard methods to obtain the number of token? Thanks in advance

Answer 1

Try this:

int frequency = new StringTokenizer(myString, " ").countTokens();

For more details see StringTokenizer.

Answer 2

Unfortunately, Scanner cannot do token counting without consuming these tokens from the input. So in order to access those tokens, you have to save them in a list:

List<String> tokens = new LinkedList<String>();
Scanner sc = new Scanner(System.in);
int count = 0;
while(sc.hasNext()) {
  tokens.add(sc.next());
  count++;
}
System.out.println("Number of tokens: "+count);

Answer 3

You can use Matcher:

Pattern pattern = Pattern.compile(token);
Matcher matcher = pattern.matcher(s);
int count = 0;
// Check all occurrences
while (matcher.find()) {
    ++count;
}

Answer 4

You could manage it using the split method:

public static int getTokenCount(String input) {
    if (input == null) {
        return 0;
    }
    input = input.trim();
    if (input.isEmpty()) {
        return 0;
    }
    return input.split("\\s+").length;
}

Answer 5

Use split(), it supports regex, unlike StringTokenizer.

int nbOfTokens = sc.nextLine().split(sc.delimiter().pattern()).length;

Answer 6

import java.io.*;
import java.util.*;
public class Solution {

    public static void main(String[] args) {
        Scanner scan = new Scanner(System.in);
        String s = scan.nextLine();
        String delims= "[ .,'!?_@]+";
        int length1=s.split(delims).length;

        System.out.println(length1);
        String[] tokens=s.split(delims);
        for(String token : tokens){
            System.out.println(token);

        }


        scan.close();
    }
}