The goal of this assignment is to create a C program which:

-takes a text file as input

-tokenizes the given input

-Parses the tokenized input to determine if it is grammatically valid

Tokenizer.c is not provided and needs to be created. Tokenizer.c will read characters from a given FILE variable and convert them into tokens. It will do so using a function defined as follows:

_Bool tokenizer(struct lexics *aLex, int *numLex, FILE *inf);

Which takes an array of type lexics, an int pointer representing the number of tokens in the input file, and a pointer to a FILE. The tokenizer function will read characters from the given FILE parameter, creating lexemes and the associated tokens. Each time a lexeme is generated, a new lexics struct will be created and the lexeme added. The generated lexeme is then tokenized, the token is added to the generated lexics struct, the lexics struct is then added to the end of the given lexics array.

The given lexical structure is free format and the location of tokens in the text file does not affect their meaning. Alphanumeric lexemes will be delimited by both whitespace and by character lexemes. Because character lexemes are used as delimiters, they cannot be constructed one token at a time. Rather the next several tokens in the file will need to be examined to determine which (if any) character lexeme is present.

The use of helper functions in the Tokenizer.c class is highly recommended. Once the tokenization process is complete, the tokenizer function should return TRUE. If there occurs an error in the process, the function should return FALSE.

Provided lexical structure:

LEFT_PARENTHESIS --> (

RIGHT_PARENTHESIS --> )

LEFT_BRACKET --> {

RIGHT_BRACKET --> }

WHILE_KEYWORD --> while

RETURN_KEYWORD --> return

EQUAL --> =

COMMA --> ,

EOL --> ;

VARTYPE --> int | void

IDENTIFIER --> [a-zA-Z][a-zA-Z0-9]*

BINOP --> + | * | != | == | %

NUMBER --> [0-9][0-9]*

The EBNF Grammar is listed below for reference

function --> header body

header --> VARTYPE IDENTIFIER LEFT_PARENTHESIS [arg-decl] RIGHT_PARENTHESIS

arg-decl --> VARTYPE IDENTIFIER {COMMA VARTYPE IDENTIFIER}

body --> LEFT_BRACKET [statement-list] RIGHT_BRACKET

statement-list --> statement {statement}

statement --> while-loop | return | assignment | body

while-loop --> WHILE_KEYWORD LEFT_PARENTHESIS expression RIGHT_PARENTHESIS statement

return --> RETURN_KEYWORD expression EOL

assignment --> IDENTIFIER EQUAL expression EOL

expression --> term {BINOP term} | LEFT_PARENTHESIS expression RIGHT_PARENTHESIS

term --> IDENTIFIER | NUMBER