LEXICAL STRUCTURE

Comments

3.7

Sub:

the ASCII

SUB

 character, also known as  control Z 

White space ( 3.6) and comments ( 3.7) can serve to separate tokens that, if

adjacent, might be tokenized in another manner. For example, the ASCII charac 

ters

 and

=

 in the input can form the operator token

 =

 ( 3.12) only if there is no

intervening white space or comment.

As a special concession for compatibility with certain operating systems, the

ASCII

SUB

 character (

\u001a

, or control Z) is ignored if it is the last character in

the escaped input stream.

Consider two tokens

x

 and

y

 in the resulting input stream. If

x

 precedes

y

,

then we say that

x

 is

to the left of

y

 and that

y

 is

to the right of

x

. For example, in

this simple piece of Java code:

class Empty {

}

we say that the

}

 token is to the right of the

{

 token, even though it appears, in this

two dimensional representation on paper, downward and to the left of the

{

 token.

This convention about the use of the words left and right allows us to speak, for

example, of the right hand operand of a binary operator or of the left hand side of

an assignment.

3.6   White Space

White space

 is defined as the ASCII space, horizontal tab, and form feed charac 

ters, as well as line terminators ( 3.4).

WhiteSpace:

the ASCII

SP

 character, also known as  space 

the ASCII

HT

 character, also known as  horizontal tab 

the ASCII

FF

 character, also known as  form feed 

LineTerminator

3.7   Comments

Java defines three kinds of

comments

:

/*

text

 */

A

traditional comment

: all the text from the ASCII

characters

/*

 to the ASCII characters

*/

 is ignored

(as in C and C++).

15