22.14

java.io.StreamTokenizer

THE PACKAGE JAVA.IO

If an alphabetic character is encountered, then an identifier is recognized, con 

sisting of that character and all following characters up to, but not including, the

first character that is neither alphabetic nor numeric, or up to end of file, which 

ever comes first. The characters of the identifier may be converted to lowercase if

this

StreamTokenizer

 is in lowercase mode.

If a comment character is encountered, then all subsequent characters are

skipped and ignored, up to but not including the next line terminator or end of file.

Then another attempt is made to recognize a token. If this

StreamTokenizer

 is

currently recognizing line terminators, then a line terminator that ends a comment

will be recognized as a token in the same manner as any other line terminator in

the contained input stream.

If a string quote character is encountered, then a string is recognized, consist 

ing of all characters after (but not including) the string quote character, up to (but

not including) the next occurrence of that same string quote character, or a line

terminator, or end of file. The usual escape sequences ( 3.10.6) such as

\n

 and

\t

are recognized and converted to single characters as the string is parsed.

If

//

 is encountered and this

StreamTokenizer

 is currently recognizing

//

comments, then all subsequent characters are skipped and ignored, up to but not

including the next line terminator or end of file. Then another attempt is made to

recognize a token. (If this

StreamTokenizer

 is currently recognizing line termi 

nators, then a line terminator that ends a comment will be recognized as a token in

the same manner as any other line terminator in the contained input stream.)

If

/*

 is encountered and this

StreamTokenizer

 is currently recognizing

/*

comments, then all subsequent characters are skipped and ignored, up to and

including the next occurrence of

*/

 or end of file. Then another attempt is made to

recognize a token.

If none of the cases listed above applies, then the only other possibility is that

the first non whitespace character encountered is an ordinary character. That char 

acter is considered to be a token and is stored in the

ttype

 field and returned.

22.14.22

public void pushBack()

Calling this method  pushes back  the current token; that is, it causes the next call

to

nextToken

 to return the same token that it just provided. Note that this method

does

not

 restore the line number to its previous value, so if the method

lineno

 is

called after a call to

pushBack

 but before the next call to

nextToken

, an incorrect

line number may be returned.

718