Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

Please write code for this program in C++ programming language. This parser will behave as follows: 1) Whitespace (spaces, tabs, newlines) used only to separate

Please write code for this program in C++ programming language.

This parser will behave as follows:

1) Whitespace (spaces, tabs, newlines) used only to separate tokens - otherwise ignored.

2) Tokens will be made up of strings of text separated by punctuation and symbols

3) Single character tokens (punctuation and symbols) may be defined.

Default behavior: treat symbols like any text character. All characters not separated by whitespace are part of the same token. Parser will provide functions (below) which allow the user to define additional single-character tokens.

Tokenizer() // this is the default class constructor - creates a parser with no initial single-character tokens (separate by whitespace " , \t, ").

Tokenizer(const vector& tokens) // class constructor - creates a parser with single-character tokens defined (defines single character tokens).

void setCharacterTokens(const vector& tokens) // resets the parser's single-character tokens to the list of tokens (separates tokens of strings into pieces when you find these characters: punctuation " !, ., \", \', , ", blocks "[, ], {, }, (, )", symbols "#, $, -")

void addCharacterTokens(const vector& newTokens) // adds the provided single-character tokens to the set of tokens this parser recognizes (separates tokens of strings into pieces when you find these characters: punctuation " !, ., \", \', , ", blocks "[, ], {, }, (, )", symbols "#, $, -")

void addCharacterToken(char newToken) // adds the provided single-character token to the set of tokens this parser recognizes (separates tokens of strings into pieces when you find these characters: punctuation " !, ., \", \', , ", blocks "[, ], {, }, (, )", symbols "#, $, -")

vector parse(string input) const // parses the string and returns a vector of tokens in string form (splits the string into pieces. piece and snippet are passed to tokenize)

vector tokenize(string snippet) // breaks a whitespace-free snippet into tokens by finding and splitting at single-character tokens (checks existence of special characters to further cut)

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image_step_2

Step: 3

blur-text-image_step3

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

More Books

Students also viewed these Databases questions

Question

14-18 Compare the two major types of planning and control tools.

Answered: 1 week ago