You are allowed to parse the input file tokens into a string in the Tokenizer methods.

To support NLP, we are going to develop a program that will parse a text file, tokenize the contents, and count the number of occurrences of each token. Here are a few things worth considering: White space is a good indicator of token end, but so is punctuation, including single and double quotations. Punctuation characters do not always indicate the end of a token. Some examples include o $3,745: the "" is a part of the token, not the end of the "$3" token o 47.5: "." Is part of the token, not the end of the "47" token o Don't:is part of the token, not the end of the "Don" token. Identical words with different capitalization are the same words: "Catwalk", "catwalk", and "CatWalk" are all the same token, so they should be counted as one token. O The Solution We are going to write a program named Prog01_Tokenization in our CSC1351 workspace that will allow a user to select a text input file to be parsed. The program will parse the file, tokenizing its content into parallel arrays: one array will contain the token, and one array will contain the token count. Java ArrayLists are not allowed for this program. The user is then prompted to choose whether or not a second input file of words to be ignored will also be input. If the user would like, you will then allow the user to select a text input file to be ignored. You can assume that the text file of ignored words, contain one word per line in ascending sorted order The user will then select a location/filename for analysis output to be stored. The program will outputa sorted list of tokens, along with the occurrence counts for each lower-case token, separated by commas As an example, an input file with the single sentence "She is earning a 6 fiure salary, but it is less than $150k." and an ignore file with the single word "but" would produce an output file with the following contents $150k,1 6,1 earning,1 figure,1 less,1 salary,1 she,1 than,1 Input/Output You will use the File Dialog box library (JFileChooser) to allow the user to browse to a place on the computer and select the input files (i.e., use the showOpenDialog method to display the dialog box and use the getSelectedFile method to retrieve the user's selection in the form of a File object). An example use of the JFileChooser library can be found on pages 335-336 of your textbook. Note that you will need the following import to use the JFileChooser library: import javax.swing.JFileChooser; You will use the File Dialog box library (3FileChooser) to allow the user to browse to a place on the computer and select the output file (i.e., use the showSaveDialog method to display the dialog box, use the getselectedFile method to retrieve the user's selection in the form of a File object, and use the File getPath() method to specify the filename selected by the user) Special Requirements Create a project in your CSC1351 workspace named "Prog01_Tokenization". Create a class named "Prog01_Tokenization" that contains your "main" method. You will create an additional class named "Tokenizer" that will provide the following public methods for use by your main program: Tokenizer(File EvalFile) Tokenizer(File EvalFile, File IgnoreFile) void OutputTokens(File OutFile) Programming Best Practices Your program should be well commented, it should be modular (i.e., use methods to keep the logic flow easy to follow, implement, and maintain), it should include user input validation, and it should employ exception management and handling. Commenting Guidelines Each class declaration should begin with comments in the following format: CSC 1351 Programming Project No Section +eauthor Each method declaration should be preceded with comments in the following format: * CSC 1351 Programming Project No Section @author +@since Member variable declarations should be immediately followed by in-line comments that describe how the member is used. Except for trivial cases, method local variables should be declared at the beginning of the method. Local variable declarations should be immediately followed by in-line comments that describe how the variable is used. Code should be well commented to describe the logic and flow of the program. It is strongly recommended that algorithm pseudocode be used as the foundation for in-code comments