COMPILER DESIGN

Detailed explanation-1: -A pattern explains what can be a token, and these patterns are defined by means of regular expressions. In programming language, keywords, constants, identifiers, strings, numbers, operators and punctuations symbols can be considered as tokens.

Detailed explanation-2: -Tokens are the terminal symbols in the grammar. Patterns are description of a class of tokens. Lexemes are words in the source program.

Detailed explanation-3: -Explanation: Parsing is used to determine whether tokens can be generated by a given grammar.

Detailed explanation-4: -A regular expression (shortened as regex or regexp; sometimes referred to as rational expression) is a sequence of characters that specifies a match pattern in text. Usually such patterns are used by string-searching algorithms for “find” or “find and replace” operations on strings, or for input validation.