Java StringTokenizer considering parentheses

Question

I am creating a program that tokenizes boolean logic expressions and returns the String array of tokens. The following is my code:

public static String[] tokenize(String s)
{
    String delims = "+";
    StringTokenizer st = new StringTokenizer(s, delims);
    String[] tokens = new String[st.countTokens()];

    int i=0;
    while (st.hasMoreElements()) {
        tokens[i++] = st.nextElement().toString();
    }

    return tokens;
}

For example, I have the following string as an input:

A+B+(C+D+(A+B))+(B+C)

Using the code I have, it will only generate the following tokens:

A
B
(C
D
(A
B))
(B
C)

Is it possible (using the same structure of code) to come up with these tokens? If not, how is it code-able?

A
B
(C+D+(A+B))
(B+C)

You could just loop through 1 character at a time, when you reach a + while not in a parenthesis, save the characters read up to there, and start a new set. The way to track if you're in a a set of parentheses is with a counter. When you hit a open parenthesis, you increment a counter by 1, when you hit a close parenthesis you decrement the counter by 1. If the counter > 0 then you're in parentheses. Oh, and if counter ever goes negative, the string is invalid, and if counter is not 0 at the end, then the string is also invalid. — Cruncher, Sep 25 '13 at 15:18

score 0 · Accepted Answer · answered Sep 25 '13 at 15:29

    ArrayList<String> tokens = new ArrayList<String>();
    String current = "";
    int counter = 0;
    for(int i = 0 ; i < input.length(); i++)
    {
        char c = input.charAt(i);
        if(counter==0 && c=='+')
        {
            tokens.add(current);
            current = "";
            continue;
        }
        else if(c=='(')
        {
            counter++;
        }
        else if(c==')')
        {
            counter--;
        }
        current += c;
    }
    tokens.add(current);

This is the solution for my comment:

You could just loop through 1 character at a time, when you reach a + while not in a parenthesis, save the characters read up to there, and start a new set. The way to track if you're in a a set of parentheses is with a counter. When you hit a open parenthesis, you increment a counter by 1, when you hit a close parenthesis you decrement the counter by 1. If the counter > 0 then you're in parentheses. Oh, and if counter ever goes negative, the string is invalid, and if counter is not 0 at the end, then the string is also invalid.

You can do the checks on counter and return false or something, to show that it is an invalid string. If you know the string is valid then this works as is. You can get an array from the ArrayList, with tokens.toArray()

Wow, that works like a charm! I wasn't so sure about the conditional statement of the initial if. Thank you for the clarification! — Janno Ramos, Sep 25 '13 at 15:36

score 0 · Answer 2 · answered Sep 25 '13 at 15:35

If you can find a simple solution go with it otherwise try mine

    String s = "A+B+(C+D+(A+B))+(B+C)";
    List<String> l = new ArrayList<>();
    StringBuilder sb = new StringBuilder();
    int p = 0;
    for (char c : s.toCharArray()) {
        if (c == '(') {
            p++;
            sb.append(c);
        } else if (c == ')') {
            p--;
            sb.append(c);
        } else if (p == 0 && c == '+') {
            l.add(sb.toString());
            sb.setLength(0);
        } else {
            sb.append(c);
        }
    }
    if (sb.length() > 0) {
        l.add(sb.toString());
    }
    System.out.println(l);

output

[A, B, (C+D+(A+B))]

Java StringTokenizer considering parentheses

2 Answers2