Java StringTokenizer with empty tokens

Question

I have a string that looks some thing like - 56,0,76,0,93,,,,,,,1230. I use a StringTokenizer to split this by ','. However, it seems that this is skipping from 93 straight to 1230. Is there a way to make it return six empty strings before it moves to 1230?

Have you checked java.util.Scanner ? – ponomandr Aug 26 '14 at 07:44 — ponomandr, Aug 26 '14 at 07:44

score 7 · Accepted Answer · answered Aug 26 '14 at 07:45

7

Use String.split() method instead.

String str = "56,0,76,0,93,,,,,,,1230";
String[] stringArr = str.split(",");

This will return an array of String.

answered Aug 26 '14 at 07:45

lxcky

1,668
2
13
26

1

this will not work if comma is in the end or beginning of the string. see my answer – Braj Aug 26 '14 at 07:55
@user3218114, actually, it does. I just tried running it. – lxcky Aug 26 '14 at 08:04
try `,,-56,0,76,0,93,,,,,,,1230,` – Braj Aug 26 '14 at 08:13

Braj · Answer 2 · 2014-08-26T07:55:25.577

3

Use String#split(regex,limit) along with limit as -1

sample code:

System.out.println(Arrays.toString(",,-56,0,76,0,93,,,,,,,1230,".split(",",-1)));

output:

[, , -56, 0, 76, 0, 93, , , , , , , 1230, ]

edited Aug 26 '14 at 07:55

answered Aug 26 '14 at 07:49

Braj

46,415
5
60
76

I agree that your answer should be the correct one. – Ralph Apr 02 '20 at 11:21

score 3 · Answer 3 · answered Aug 26 '14 at 07:51

StringTokenizer has a constructor which takes 3 arguments which the third is a boolean that controls if the delimiters are to be returned as tokens or not. You should set it to true.

new StringTokenizer(yourString, yourDelimiters, true);

Now the delimiters are returned too, and it is easy to tell if an empty string exists by checking if two consecutive tokens are of delimiters.

You can extend the tokenizer or write a wrapper class of your own for reuse purposes.

score 3 · Answer 4 · answered Feb 12 '18 at 17:38

Please find a working copy to overcome Empty Token

public class TestStringTokenStrict {

/**
 * Strict implementation of StringTokenizer
 * 
 * @param str
 * @param delim
 * @param strict
 *            true = include NULL Token
 * @return
 */
static StringTokenizer getStringTokenizerStrict(String str, String delim, boolean strict) {
    StringTokenizer st = new StringTokenizer(str, delim, strict);
    StringBuffer sb = new StringBuffer();

    while (st.hasMoreTokens()) {
        String s = st.nextToken();
        if (s.equals(delim)) {
            sb.append(" ").append(delim);
        } else {
            sb.append(s).append(delim);
            if (st.hasMoreTokens())
                st.nextToken();
        }
    }
    return (new StringTokenizer(sb.toString(), delim));
}

static void altStringTokenizer(StringTokenizer st) {
    while (st.hasMoreTokens()) {
        String type = st.nextToken();
        String one = st.nextToken();
        String two = st.nextToken();
        String three = st.nextToken();
        String four = st.nextToken();
        String five = st.nextToken();

        System.out.println(
                "[" + type + "] [" + one + "] [" + two + "] [" + three + "] [" + four + "] [" + five + "]");
    }
}

public static void main(String[] args) {
    String input = "Record|One||Three||Five";
    altStringTokenizer(getStringTokenizerStrict(input, "|", true));
}}

Output will be

[Record] [One] [ ] [Three] [ ] [Five]

score 1 · Answer 5 · answered Aug 26 '14 at 07:44

1

Use StringTokenizer(String str, String delim, boolean returnDelims) with returnDelims set to true

If the returnDelims flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a string of length one.

answered Aug 26 '14 at 07:44

Adam Siemion

15,569
7
58
92

score 1 · Answer 6 · answered Aug 26 '14 at 07:49

1

String.split(",",-1); Without the -1 it will ignore trailing delimiter

answered Aug 26 '14 at 07:49

coffeeaddict

858
5
3

Even if you don't put -1, it won't ignore the trailing delimiter. – lxcky Aug 26 '14 at 08:06

score 1 · Answer 7 · answered Oct 07 '22 at 18:19

I have this answer.. which is similar to the previous one..it has no bugs as far as I know.. based on using StringTokenizer with true arguments at the end.

// inputs:  text, delimiters
    Vector tokens;
    tokens = new Vector();
    boolean lastissep = false;
    tokenizer = new StringTokenizer(text, delimeters,true);
    while (tokenizer.hasMoreTokens()) {
        Object o = tokenizer.nextElement();
        if (((String)(o)).equals(delimeters))
        {
            tokens.addElement("");
            lastissep = true;
        }
        else
        {
            tokens.addElement(o);
            lastissep = false;
            if (tokenizer.hasMoreTokens()) 
            {
                tokenizer.nextElement();
                lastissep = true;
            }   
        }   
          
    }
    if (lastissep)
        tokens.addElement("");

score 0 · Answer 8 · answered Aug 26 '14 at 08:11

Use Google's Guava lib Splitter

String str = "56,0,76,0,93,,,,,,,1230";
Iterable<String> splitResult = Splitter.on(",").split(str);

and transform to a ArrayList

ArrayList<String> elementList = Lists.newArrayList(splitResult);

elementList.size(); // 12

Java StringTokenizer with empty tokens

8 Answers8

Linked

Related