TOKENS in C++

The smallest individual units in program are known as tokens. A C++ program is written using these tokens, white spaces and the syntax of the language. C++ has the following tokens.

1. Keywords
2. Identifiers
3. Constants
4. Strings
5. Operators

1. Keywords:

The keywords implement specific C++ language feature. They are explicitly reserved identifiers and can’t be used as names for the program variables or other user defined program elements.

ISO C++ (C++98) Keywords

Note: Old compilers may not recognize some or all of the above keywords

2. Identifiers:

Identifiers refers to the name of variable , functions, array, class etc. created by programmer. Each language has its own rule for naming the identifiers. The following are common rules for both C and C++ to define identifiers.

1. Only alphabetic chars, digits and under score are permitted.
2. The name can’t start with a digit.
3. Upper case and lower case letters are distinct.
4. A declared keyword can’t be used as a variable name
5. In ANSI C the maximum length of a variable is 32 chars but in c++ there is no bar.

Examples of valid / invalid identifiers:

1. a, x, n, num, SUM, fact, grand_total, sum_of_digits, sum1 are valid.
2. ab Ab aB AB are treated differently.
3. $amount, ‘num’, grand-total, sum of digits, 4num are invalid.

3. Constants:

Constants refer to values that do not change during the execution of a program. Constants can be divided into two major categories:

1. Numeric constants
   • Integer constants
     • Ex: 16, 32768U, 020, 0x10, 012L
   • floating point-Real constants
     • Ex: 5.19, 0.12E+2, 7.5e-1, 0.00004, 12e0
2. Character Constants
   • Single character constants
     • Ex: ‘A’ , ‘a’, ‘ ‘, ‘0’, ‘\0’
   • String constants
     • Ex: “Hello!”

Escape Sequences or Backslash Character Constants in C++ : C language supports some non-printable characters, as well as backslash ( \ ) which can be expressed as escape sequences. An escape sequence always starts with backslash followed by one or more special characters. For example, a new line character is represented “\n” or endl. These are used in formatting output screen, i.e. escape sequence are used in output functions. Some escape sequences are given below:

4. Strings:

You already know string constants, which were introduced for text output using the cout stream. A string constant consists of a sequence of characters enclosed in double quotes. Ex: “Today is a beautiful day!”.

A string constant is stored internally without the quotes but terminated with a null character, \0, represented by a byte with a numerical value of 0 — that is, all the bits in this byte are set to 0. Thus, a string occupies one byte more in memory than the number of characters it contains. An empty string, “”, therefore occupies a single byte. The terminating null character( ‘\0’ ) is not the same as the number zero and has a different character code than zero. Thus, the string “0” comprises two bytes, the first byte containing the code for the character zero 0 (ASCII code 48) and the second byte the value 0.

Note: C++ supports two types of string representation- the C-Style (Character style) and string class type introduced with standard c++.

5. Operators

An operator is a symbol which represents a particular operation that can be performed on data. An operand is the object on which an operation is performed.C operators can be classified as:

1. Arithmetic operators
2. Relational operators
3. Logical operators
4. Assignment operators
5. Increment or Decrement operators
6. Conditional operator
7. Bit wise operators
8. unary operator
9. Special operators
10. Additional operators in c++