COMPILER DESIGN

Detailed explanation-1: -In a compiler, keyboards of a language are recognized during the lexical analysis of the program. Typically, the lexical analysis phase of compilation breaks the input text up into sequences of lexemes that each belongs to some particular token type that’s useful in later analysis.

Detailed explanation-2: -The lexical analyzer also follows rule priority where a reserved word, e.g., a keyword, of a language is given priority over user input. That is, if the lexical analyzer finds a lexeme that matches with any existing reserved word, it should generate an error.

Detailed explanation-3: -The lexical analyzer is responsible for removing the white spaces and comments from the source program. It corresponds to the error messages with the source program. It helps to identify the tokens. The input characters are read by the lexical analyzer from the source code.

Detailed explanation-4: -Tokens are defined often by regular expressions, which are understood by a lexical analyzer generator such as lex. The lexical analyzer (generated automatically by a tool like lex, or hand-crafted) reads in a stream of characters, identifies the lexemes in the stream, and categorizes them into tokens.

Detailed explanation-5: -The first phase of the compiler is the lexical analyzer, also known as the scanner, which recognizes the basic language units, called tokens. The exact characters in a token is called its lexeme.