Stack Overflow
Stack Overflow

Questions tagged [stringtokenizer]

StringTokenizer is a Java class that breaks a string into tokens.

This tag is used for questions about or involving the StringTokenizer class in Java, which is used to break a single string into multiple, individual "tokens".

StringTokenizer is an old class, called the "legacy class" in documentation, suggesting to use String.split instead. However as of 1.7, it is not deprecated. Another class with similar but more advanced functionality is .

Questions tagged should also be tagged .

Useful links:

627 questions
2
votes
4 answers

Passing multiple delimiters to StringTokenizer constructor

I have seen that the syntax for passing multiple delimiters (eg. '.' , '?', '!') to the StringTokenizer constructor is: StringTokenizer obj=new StringTokenizer(str,".?!"); What I am not getting is that, I have enclosed all the delimiters together…
user11745382
2
votes
4 answers

Java How to Convert Token to String?

Cant seem to figure out how to convert a Token (StringTokenizer) into a String. I have a String[] of keywords, and I read in multiple lines of text from a text file. StringTokenizer is used to chop the sentence up,,, at which point I need to pass…
Donal.Lynch.Msc
  • 3,365
  • 12
  • 48
  • 78
2
votes
6 answers

StringTokenizer showing unwanted results

When I ran the following code I found something strange. The output of below program is token1=AAAAA token2=BBBBB| However, From my understanding, it should be token1=AAAAA token2=BBBBB|DUMMY public class TestToken { public static void…
T-Bag
  • 10,916
  • 3
  • 54
  • 118
2
votes
0 answers

span_tokenize gives generator object as output

I have written a simple piece of code to see exactly how the span_tokenize function works. Documentation for this can be found here: http://www.nltk.org/api/nltk.tokenize.html Here is my piece of code import nltk from nltk.tokenize.api import…
Shardul Pendharkar
  • 137
  • 1
  • 11
2
votes
1 answer

How do you create a word frequency counter in Java without hashmap or arrays?

I have been trying to create a program in Java that outputs each word found in a string and the number of times they appear. I have been trying to do this without using any arrays or hashmap, and by only using string manipulation methods and string…
fizzle
  • 23
  • 3
2
votes
3 answers

unable to understand Output of the following simple program

/* A Java program to illustrate working of StringTokenizer class:*/ import java.util.*; public class NewClass { public static void main(String args[]) { System.out.println("Using Constructor 3 - "); StringTokenizer st3 = …
Ashutosh Tiwari
  • 311
  • 2
  • 8
2
votes
1 answer

countTokens() method misbehave when using directly in loop

countTokens() method returns 3 instead of 5 when i am using in for loop. public static void main(String[] args) { String s = "foo1,foo2,foo3,foo4,foo5"; StringTokenizer tokenizer = new StringTokenizer(s, ","); …
user19951215
  • 111
  • 5
2
votes
2 answers

Parse string recursively with StringTokenizer

I am trying to parse a string recursively with StringTokenizer. The string represents a tree, in the form: [(0,1),[(00,01,02),[()],[()]]] where information of the node is stored inside the parenthesis, while the brackets are the children of a node,…
GianniPele
  • 41
  • 1
  • 1
  • 7
2
votes
1 answer

How to get the delimiter from a StringTokenizer?

StringTokenizer tokenizer = new StringTokenizer(s, " ,.:;?![]'"); Is there a way to also retrieve the delimiter, in this case all thee punctuation marks? For example, "This is a test, and is that a test too?" I want the result of tokenization also…
user697911
  • 10,043
  • 25
  • 95
  • 169
2
votes
3 answers

Tokenizer java (string)

For example, Ive got Some stringa.b.c.d`, I have separated it by tokens. but now I want to do a specific thing with a, then something else with b, and something else with c. How can I do that? String value="a.b.c.d"; StringTokenizer tokenize = new…
mica1234
  • 27
  • 5
2
votes
0 answers

Tokenizer function using Boost::qi and Boost::lex throws "stack overflow" exception for large inputs

This function identifies words and stores them into a vector of wstrings. It worked fine for test cases like: this.is://a-test HI,hi But for input string exceeding 60 tokens or above it throws me stack overflow exception and program breaks,I suspect…
Divya Ravi
  • 21
  • 4
2
votes
2 answers

How not count blank spaces in StringTokenizer

I have this line StringTokenizer tk=new StringTokenizer (b=10," =",true); my output is b = 10 and this perfect but again if my line is this StringTokenizer tk=new StringTokenizer (DEFI b=10," =",true); And my output is DEFI b = 10 I don't wanna…
Baker1562
  • 337
  • 3
  • 20
2
votes
1 answer

Ignore special characters on StringTokenizer

I want to ignore special characters like , " ; : using string tokenizer. For example, if I enter: He said, "That's not a good idea." output should be: He Said that s not a good idea This is my current code class MyClass { public static void…
nascar895
  • 149
  • 1
  • 12
2
votes
0 answers

"The method addAttribute(Class) is undefined for the type TokenStream" while using Apache Lucene in Java

I am using Apache Lucene for tokenizing string in Java and I am encountering the below error even after importing all the external jar files. The error is: The method addAttribute(Class) is undefined for the type TokenStream The line of code in…
Akash Upase
  • 21
  • 3
2
votes
2 answers

Retrieving TextView.getText() to set a CountDownTimer with a Start Button

I am in the dawn of my Android programming adventure and have just become able to communicate between on-screen views. So the next step for me is to successfully pull text from a TextView (set by a Dialog) and use a Start Button to run a Timer based…
Stembolt
  • 37
  • 4
1 2 3
…
41 42