THE PACKAGE JAVA.UTIL

java.util.StringTokenizer

21.10

21.10 The Class

java.util.StringTokenizer

The

StringTokenizer

 class provides a way to break a

String

 into tokens. The

tokenizing method used by this class is much simpler than the one used by the

class

java.io.StreamTokenizer

. For example, a

StringTokenizer

 does not

distinguish among identifiers, numbers, and quoted strings; moreover, it does not

recognize and skip comments.

A

StringTokenizer

 can serve as an

Enumeration

 ( 21.1).

public class StringTokenizer implements Enumeration {

public StringTokenizer(String str, String delim,

boolean returnTokens);

public StringTokenizer(String str, String delim);

public StringTokenizer(String str);

public boolean hasMoreTokens();

public String nextToken();

public String nextToken(String delim);

public boolean hasMoreElements();

public Object nextElement();

public int countTokens();

}

A

StringTokenizer

 simply divides characters into classes: delimiters and

other characters. The tokenizer behaves in one of two ways, depending on

whether it was created with

returnTokens

 having the value

true

 or

false

.

If

returnTokens

 is

false

, delimiter characters merely serve to separate

tokens of interest. A token is thus a maximal sequence of consecutive characters

that are not delimiters.

If

returnTokens

 is

true

, delimiter characters are themselves considered to

be tokens of interest. A token is thus either one delimiter character or a maximal

sequence of consecutive characters that are not delimiters.

A

StringTokenizer

 internally maintains a current position within the

String

 to be tokenized. Some operations advance this current position past the

characters processed.

A token is returned by taking a substring ( 20.12.32) of the string that was

used to create the

StringTokenizer

.

651