Using full stop as string delimiter in Stringtokenizer

Question

I would like to exclude full stop . from string tokens like in the following case:

 I go to school.

Would like to generate tokens as:

        [ I] [go] [to] [school]

I used stringTokenizer as:

List<String> listStr = new ArrayList<String>();
StringTokenizer strToken = new StringTokenizer(str);                    
int i =0;
while(strToken.hasMoreTokens()){
    listStr.add(i, strToken.nextToken());
    i=i+1;
}

But the last token is school. which i don;t want. How would i do it?

using a regex with `replace()` or `replaceAll()` method should do, or have I misunderstood the questions? — posdef, May 08 '12 at 13:26
Btw, you increment `i` is not needed. Just use the `add(Object)` and it will automatically be added to the end of your list — Guillaume Polet, May 08 '12 at 13:26

score 5 · Answer 1 · answered May 08 '12 at 13:25

5

Isn't this simpler?

List<String> list = Arrays.asList("I go to school.".split("[ .]"));

System.out.println(list) prints

[I, go, to, school]

answered May 08 '12 at 13:25

adarshr

61,315
23
138
167

1

Yes it is simpler. Else, if he/she wants to keep the StringTokenizer, use this constructor: `new StringTokenizer(str," \t\f\r\n.");` – Guillaume Polet May 08 '12 at 13:28

score 2 · Accepted Answer · edited Jan 13 '13 at 21:15

2

The StringTokenizer has a constructor that takes another argument: The characters that act as delimiters. The following code illustrates this:

StringTokenizer tokenizer = new StringTokenizer("I go to school.", " \t\r\n.");
while(tokenizer.hasMoreElements()) {
    System.out.println(tokenizer.nextElement());
}

This is the output, which should be what you want:

I
go
to
school

edited Jan 13 '13 at 21:15

Audrius Meškauskas

20,936
12
75
93

answered May 08 '12 at 13:28

rolve

10,083
4
55
75

score 1 · Answer 3 · answered May 08 '12 at 13:25

1

You can do this on the initial string: string.replace(".", "") then do the tokenizer.

answered May 08 '12 at 13:25

AHungerArtist

9,332
17
73
109

score 1 · Answer 4 · answered May 08 '12 at 13:29

1

StringTokenizer can take a lot of delimiters, so now in your case you should build the strink tokenizer the following way:

new StringTokenizer(yourString, " .");

And thats it.

Hope it helps.

answered May 08 '12 at 13:29

Juan Alberto López Cavallotti

4,644
3
25
43

Using full stop as string delimiter in Stringtokenizer

4 Answers4