COMPILER DESIGN

Detailed explanation-1: -Tokens are the smallest elements of a program, which are meaningful to the compiler. The following are the types of tokens: Keywords, Identifiers, Constant, Strings, Operators, etc.

Detailed explanation-2: -A token is the smallest unit in programs. Keywords are predefined or reserved words that have their own importance. The main purpose of constant is to make the value fix. In C, identifiers are user defined words. Mainly used for naming variables, functions, arrays, structures etc.

Detailed explanation-3: -Tokens are some of the most important elements used in the C language for creating a program. One can define tokens in C as the smallest individual elements in a program that is meaningful to the functioning of a compiler. A token is the smallest unit used in a C program.

Detailed explanation-4: -Tokens. Lexemes are said to be a sequence of characters (alphanumeric) in a token. There are some predefined rules for every lexeme to be identified as a valid token. These rules are defined by grammar rules, by means of a pattern.