Question
Please write code for this program in C++ programming language. This parser will behave as follows: 1) Whitespace (spaces, tabs, newlines) used only to separate
Please write code for this program in C++ programming language.
This parser will behave as follows:
1) Whitespace (spaces, tabs, newlines) used only to separate tokens - otherwise ignored.
2) Tokens will be made up of strings of text separated by punctuation and symbols
3) Single character tokens (punctuation and symbols) may be defined.
Default behavior: treat symbols like any text character. All characters not separated by whitespace are part of the same token. Parser will provide functions (below) which allow the user to define additional single-character tokens.
Tokenizer() // this is the default class constructor - creates a parser with no initial single-character tokens (separate by whitespace " , \t, ").
Tokenizer(const vector
void setCharacterTokens(const vector
void addCharacterTokens(const vector
void addCharacterToken(char newToken) // adds the provided single-character token to the set of tokens this parser recognizes (separates tokens of strings into pieces when you find these characters: punctuation " !, ., \", \', , ", blocks "[, ], {, }, (, )", symbols "#, $, -")
vector
vector
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started