Please use C with multithread and multi-process. Thank you so much!

In this project, you will design and implement a client-server based synchronized and parallel keyword search application where a single server will receive search requests including a and a from multiple clients, and will search the keyword in all the files located in the specified directory. Client and server will communicate using message queues. Multiple client pro- cesses will be able to talk to the server at the same time. This project will enable you to reuse some components that you implemented in project 2, and practice with additional topics including the pro- ducer/consumer model (through a bounded buffer), semaphores, and readers/writers locks. For this project, you are allowed to work in groups of at most 2 members. No more than two students can work together; however, if you prefer to work alone, then you are allowed to do so. Each group will make a single submission. 1.1 Client Program The client will issue search requests to the server. The name of the client program must be spks_client (synchronized parallel keyword search) and it will be invoked as follows: ./spks_client The is the name of the input file including all the search requests to be directed to the server. The client process will get the search requests from the input file line by line (one request at a time) and will send these requests to the server through the request message queue created by the server. The format of the search requests included in the input file will be as follows: You can make the following assumptions about the : 1. The is a full path not exceeding the MAXDIRPATH characters 2. The is a single word not exceeding the MAXKEYWORD characters, enclosed by whitespace characters not including any whitespace characters inside. Note that a whitespace character can be either a , or a , or a character. A sample input file content can be as follows: /homealtipar/dirOne is /homealtipar/dirOne a /homealtipar/dir Two a A client process that is invoked with the command line argument will first attach to the server's message queue identified by a key derived from a file pathname and a nonzero integer value. The server will use this message queue to receive requests from the client processes. Since you are the programmer of both the client and the server, you will decide this pathname and the integer value that the ftok function expects to uniquely identify a message queue. For instance, you can use the name of the server program and the integer 1 in both client and server to create a key as follows: ftok ("spks_server.c", 1) See the man page of the ftok for more details on obtaining a key. You can also check the man page of the msgget function to learn more about how to get a message queue identifier using the key returned by the ftok function. Some other useful functions include msgget, msgsnd, msgrev, msgctl. Check the System V Message Queues example and the tutorial link pro- vided in the class website for more information about message queues. After attaching the server's message queue, the client will generate a separate request message for every search request included in the . The generated message will be sent to the server via this message queue. The server must have created the master message queue earlier waiting for client requests to arrive. Therefore, the server must be started earlier than the clients. Several client processes can send requests to the server using the same message queue. A special command will be used to terminate the server. When the server receives this special search command, then it will terminate and will not be able to serve any further requests. This special command will only include "exit" in the message as follows: exit 1.2 Server Program The server will accept search requests from several clients and for each request, it will search the in all the files sitting in the given using multiple threads. The server program must be named spks_server and it will be invoked as follows: ./spks_server The server process should be started before the client processes. It will immediately create a message queue that will be used to receive client requests. Similar to clients, a key will be necessary to create this message queue. Actually, clients and the server should exactly use the same key so that they can refer the same message queue. Therefore, the parameters used in ftok function should be the same in both clients and the server. However, only server should use the IPC_CREAT option in msgget function to create the message queue and wait for the client requests to arrive. Clients will not need to re-create a message queue, they will only join this message queue that is previously created by the server. Again, check the System V Message Queues examples and the tutorial provided in the class website for more information. Quaff. oh quaff this kind nepenthe and forger this lost Lenore!" Quoth the Raven "Nevermore." Worker A sample input file content can be as follows: /homealtipar/dirOne is /homealtipar/dirOne a /homealtipar/dirTwo a Bounded Buffer Then, the printer thread will print the following to the output file: In this project, you will design and implement a client-server based synchronized and parallel keyword search application where a single server will receive search requests including a and a from multiple clients, and will search the keyword in all the files located in the specified directory. Client and server will communicate using message queues. Multiple client pro- cesses will be able to talk to the server at the same time. This project will enable you to reuse some components that you implemented in project 2, and practice with additional topics including the pro- ducer/consumer model (through a bounded buffer), semaphores, and readers/writers locks. For this project, you are allowed to work in groups of at most 2 members. No more than two students can work together; however, if you prefer to work alone, then you are allowed to do so. Each group will make a single submission. filel.txt:1: Then, methought the air grew denser, perfumed from an unseen censer filel.txt:2:Swung by Seraphim whose foot-falls tinkled on the tufted floor. filel.txt:6:Quoth the Raven "Nevermore." Input File Client Message Queue Request Queue IM Server Input File A client process that is invoked with the command-line argument will first attach to the server's message queue identified by a key derived from a file path name and a nonzero integer value. The server will use this message queue to receive requests from the client processes. Since you are the programmer of both the client and the server, you will decide this pathname and the integer value that the ftok function expects to uniquely identify a message queue. For instance, you can use the name of the server program and the integer 1 in both client and server to create a key as follows: ftok ("spks_server.c", 1) See the man page of the ftok for more details on obtaining a key. You can also check the man page of the msgget function to learn more about how to get a message queue identifier using the key returned by the ftok function. Some other useful functions include msgget, msgsnd, msgrev, msgctl. Check the System V Message Queues example and the tutorial link pro- vided in the class website for more information about message queues. After attaching the server's message queue, the client will generate a separate request message for every search request included in the . The generated message will be sent to the server via this message queue. The server must have created the master message queue earlier waiting for client requests to arrive. Therefore, the server must be started earlier than the clients. Several client processes can send requests to the server using the same message queue. A special command will be used to terminate the server. When the server receives this special search command, then it will terminate and will not be able to serve any further requests. This special command will only include "exit" in the message as follows: Client (Child The server will perform the search requests received from the message queue as follows. First. it will immediately create a new child process to serve each request and the search information will be passed to this child process. The child process will then create a number of threads to serve the search request. Number of threads be created by the child process depends on the number of files located inside the . If there are N files inside the , then the child process will create N + 1 threads: N worker threads and 1 printer thread. Each worker thread will be responsible from a different search file, it will scan the search file and look for the matching lines based on the . Your program will only search the first level files in the given . If the includes any sub-directories, you should skip them without searching the files in these sub-directories. A match in a line will be an exact match of an entire word only and the maximum length of a line in a search file will never exceed MAXLINESIZE. When a worker thread finds a match, it will prepare an item that includes the following information: the name of the file where the match has occurred, the line number of the match, and the line itself (i.e. the line string excluding the newline character at the end). If the keyword is seen, for example, in 5 separate lines, then 5 separate items will be created. If the keyword is seen more than once in the same line, then a single item will be created for that line. Whenever an item is created, then the worker thread will add this item to a memory buffer: a bounded buffer. The command-line argument passed to the server (buffersize>) specifies the size of the bounded buffer to be used by the threads. Each worker thread will add the produced items to this bounded buffer. The buffer can hold at most items. This bounded buffer can be implemented in one of two ways: 1) as a linked list of items; or 2) as a circular array of pointers to items. You can choose either one of these implementation options. A worker thread will add a new item to the end of the buffer. The buffer has to be accessed by all worker threads. While trying to insert an item, if a worker thread finds the buffer full, then the thread has to go into sleep (blocked) mode until there is space for one more item. Since all worker threads will access the buffer concurrently, the access must be coordinated. Otherwise we might end up with a corrupted buffer. You will use semaphores to protect the buffer and to synchronize the threads so that they can sleep and wake- up when necessary. Check the sem-pc.e example provided in the class website as an example on implementing these issues. Beside the worker threads, printer thread will also access the bounded buffer. The job of the printer thread will be to retrieve the items from the bounded buffer and print them into an output file. While the items are added to the buffer by the worker threads, the printer thread can work concurrently and try to retrieve the items from the buffer and print them to the output file. The printer thread will try to retrieve one item from the bounded buffer at a time. If the buffer is empty, the printer thread has to go into the sleeping mode until the buffer has at least one item. When the printer thread is successful in retrieving an item from the buffer, it will print it to the output file in the following format: 1.1 Client Program The client will issue search requests to the server. The name of the client program must be spks_client (synchronized parallel keyword search) and it will be invoked as follows: DO File Worker - . Bounded Buffer File Worker Child ./spks_client Figure 1: System Structure The output file will be named as output.txt and all child processes (only the printer thread from each child process) will write their output into this single output file. Since the output file can be modified by multiple processes concurrently, you should also synchronize the access to this output file. For this purpose, you will use the font 1 system call available in Unix providing file locking mechanism. Note that advisory locking should be sufficient for your case and you can assume that the maximum length of an output line will never exceed MAXOUTSIZE You do not need to worry about the ordering of the lines in the output file; before testing your output, we will sort the output file ourselves. Check the file_locking_solution.c example and the tutorial provided in the class website to see an example use of file locking. Pay attention to the following issues while developing your application: The output should not contain the name of the directory, just the name of the file. . Only the lines that include the exact match of an entire word is printed. If the keyword is seen more than once in a line, then a single output will be printed for that line. For example, if there was a line as "a a a". then the number of matches on this line would be 3 for the keyword "a". However, it will be reported only once. . For each search request received from the request queue, a specific bounded buffer, a specific printer thread, and bunch of worker threads (one for each file in the given directoryPath>) will be created by the child process that is assigned for that search request. . Your program will only search the first level files in the given All the worker threads for a given search request will do the same thing on different files and will fill the same bounded buffer. Your program will pass rece request to a child process and hand the next search request immediately without waiting for the created child to complete its execution. However, you should also make sure that your program does not terminate before printing all the outputs of the already received requests. To achieve this, you count the number of children created by the parent and after handling all the requests (exit request is received), then the parent process can call wait() system call in a loop for count times. We will make sure that the exit request is sent as the last request in the queue while testing your project. You need to make sure that the created child process in the server does not terminate before its worker and printer threads are done. For this purpose, first you need to join all your worker threads, then insert a dummy item to the bounded buffer to indicate the end of search, and then join your printer thread. A dummy item can be one with a negative line number, and once the printer threads receives this dummy item, it can terminate. For the thread joins to work properly. make sure your threads are set as joinable using PTHREAD_CREATE_JOINABLE. Figure 1 plots the general structure of this application: 2 Constants and Tips The is the name of the input file including all the search requests to be directed to the server. The client process will get the search requests from the input file line by line (one request at a time) and will send these requests to the server through the request message queue created by the server. The format of the search requests included in the input file will be as follows: exit 1.2 Server Program The server will accept search requests from several clients and for each request, it will search the in all the files sitting in the given using multiple threads. The server program must be named spks_server and it will be invoked as follows: MAXDIRPATH will be 1024 MAXKEYWORD will be 256 MAXLINESIZE will be 1024 MAXOUTSIZE will be 2048 Remember to use thread-safe library functions inside your threads! (for instance strtok_r() instead of strtok()). . While testing your application, remember to clean the previously created message queues be- fore restarting your application. In order to clean these system resources manually, run the clean_os.sh script provided in the class website. Some of the examples posted on the class website will be very useful for this project. ./spks_server You can make the following assumptions about the : filename:linenumber:linestring For example, if the keyword is "the" and the content of the file to be searched (assume the file name is filel.txt) is as follows: 3 Useful Clean-up Commands 1. The is a full path not exceeding the MAXDIRPATH characters 2. The is a single word not exceeding the MAXKEYWORD characters, enclosed by whitespace characters not including any whitespace characters inside. Note that a whitespace character can be either a , or a , or a character. The server process should be started before the client processes. It will immediately create a message queue that will be used to receive client requests. Similar to clients, a key will be necessary to create this message queue. Actually, clients and the server should exactly use the same key so that they can refer the same message queue. Therefore, the parameters used in ftok function should be the same in both clients and the server. However, only server should use the IPC_CREAT option in msgget function to create the message queue and wait for the client requests to arrive. Clients will not need to re-create a message queue, they will only join this message queue that is previously created by the server. Again, check the System V Message Queues examples and the tutorial provided in the class website for more information. Then, methought, the air grew denser, perfumed from an unseen censer Swung by Seraphim whose foot-falls tinkled on the rufted floor "Wretch, "I cried, "thy God hath lent thee-by these angels he hath sent thee Respite-respite and nepenthe from thy memories of Lenore; While testing your project, your server might create multiple child processes and terminate un- expectedly due to a segmentation fault. At that point, you might have several zombie processes existing in the system. In order to check current processes in the system including the "spks" name in it, you can run the following command: In this project, you will design and implement a client-server based synchronized and parallel keyword search application where a single server will receive search requests including a and a from multiple clients, and will search the keyword in all the files located in the specified directory. Client and server will communicate using message queues. Multiple client pro- cesses will be able to talk to the server at the same time. This project will enable you to reuse some components that you implemented in project 2, and practice with additional topics including the pro- ducer/consumer model (through a bounded buffer), semaphores, and readers/writers locks. For this project, you are allowed to work in groups of at most 2 members. No more than two students can work together; however, if you prefer to work alone, then you are allowed to do so. Each group will make a single submission. 1.1 Client Program The client will issue search requests to the server. The name of the client program must be spks_client (synchronized parallel keyword search) and it will be invoked as follows: ./spks_client The is the name of the input file including all the search requests to be directed to the server. The client process will get the search requests from the input file line by line (one request at a time) and will send these requests to the server through the request message queue created by the server. The format of the search requests included in the input file will be as follows: You can make the following assumptions about the : 1. The is a full path not exceeding the MAXDIRPATH characters 2. The is a single word not exceeding the MAXKEYWORD characters, enclosed by whitespace characters not including any whitespace characters inside. Note that a whitespace character can be either a , or a , or a character. A sample input file content can be as follows: /homealtipar/dirOne is /homealtipar/dirOne a /homealtipar/dir Two a A client process that is invoked with the command line argument will first attach to the server's message queue identified by a key derived from a file pathname and a nonzero integer value. The server will use this message queue to receive requests from the client processes. Since you are the programmer of both the client and the server, you will decide this pathname and the integer value that the ftok function expects to uniquely identify a message queue. For instance, you can use the name of the server program and the integer 1 in both client and server to create a key as follows: ftok ("spks_server.c", 1) See the man page of the ftok for more details on obtaining a key. You can also check the man page of the msgget function to learn more about how to get a message queue identifier using the key returned by the ftok function. Some other useful functions include msgget, msgsnd, msgrev, msgctl. Check the System V Message Queues example and the tutorial link pro- vided in the class website for more information about message queues. After attaching the server's message queue, the client will generate a separate request message for every search request included in the . The generated message will be sent to the server via this message queue. The server must have created the master message queue earlier waiting for client requests to arrive. Therefore, the server must be started earlier than the clients. Several client processes can send requests to the server using the same message queue. A special command will be used to terminate the server. When the server receives this special search command, then it will terminate and will not be able to serve any further requests. This special command will only include "exit" in the message as follows: exit 1.2 Server Program The server will accept search requests from several clients and for each request, it will search the in all the files sitting in the given using multiple threads. The server program must be named spks_server and it will be invoked as follows: ./spks_server The server process should be started before the client processes. It will immediately create a message queue that will be used to receive client requests. Similar to clients, a key will be necessary to create this message queue. Actually, clients and the server should exactly use the same key so that they can refer the same message queue. Therefore, the parameters used in ftok function should be the same in both clients and the server. However, only server should use the IPC_CREAT option in msgget function to create the message queue and wait for the client requests to arrive. Clients will not need to re-create a message queue, they will only join this message queue that is previously created by the server. Again, check the System V Message Queues examples and the tutorial provided in the class website for more information. Quaff. oh quaff this kind nepenthe and forger this lost Lenore!" Quoth the Raven "Nevermore." Worker A sample input file content can be as follows: /homealtipar/dirOne is /homealtipar/dirOne a /homealtipar/dirTwo a Bounded Buffer Then, the printer thread will print the following to the output file: In this project, you will design and implement a client-server based synchronized and parallel keyword search application where a single server will receive search requests including a and a from multiple clients, and will search the keyword in all the files located in the specified directory. Client and server will communicate using message queues. Multiple client pro- cesses will be able to talk to the server at the same time. This project will enable you to reuse some components that you implemented in project 2, and practice with additional topics including the pro- ducer/consumer model (through a bounded buffer), semaphores, and readers/writers locks. For this project, you are allowed to work in groups of at most 2 members. No more than two students can work together; however, if you prefer to work alone, then you are allowed to do so. Each group will make a single submission. filel.txt:1: Then, methought the air grew denser, perfumed from an unseen censer filel.txt:2:Swung by Seraphim whose foot-falls tinkled on the tufted floor. filel.txt:6:Quoth the Raven "Nevermore." Input File Client Message Queue Request Queue IM Server Input File A client process that is invoked with the command-line argument will first attach to the server's message queue identified by a key derived from a file path name and a nonzero integer value. The server will use this message queue to receive requests from the client processes. Since you are the programmer of both the client and the server, you will decide this pathname and the integer value that the ftok function expects to uniquely identify a message queue. For instance, you can use the name of the server program and the integer 1 in both client and server to create a key as follows: ftok ("spks_server.c", 1) See the man page of the ftok for more details on obtaining a key. You can also check the man page of the msgget function to learn more about how to get a message queue identifier using the key returned by the ftok function. Some other useful functions include msgget, msgsnd, msgrev, msgctl. Check the System V Message Queues example and the tutorial link pro- vided in the class website for more information about message queues. After attaching the server's message queue, the client will generate a separate request message for every search request included in the . The generated message will be sent to the server via this message queue. The server must have created the master message queue earlier waiting for client requests to arrive. Therefore, the server must be started earlier than the clients. Several client processes can send requests to the server using the same message queue. A special command will be used to terminate the server. When the server receives this special search command, then it will terminate and will not be able to serve any further requests. This special command will only include "exit" in the message as follows: Client (Child The server will perform the search requests received from the message queue as follows. First. it will immediately create a new child process to serve each request and the search information will be passed to this child process. The child process will then create a number of threads to serve the search request. Number of threads be created by the child process depends on the number of files located inside the . If there are N files inside the , then the child process will create N + 1 threads: N worker threads and 1 printer thread. Each worker thread will be responsible from a different search file, it will scan the search file and look for the matching lines based on the . Your program will only search the first level files in the given . If the includes any sub-directories, you should skip them without searching the files in these sub-directories. A match in a line will be an exact match of an entire word only and the maximum length of a line in a search file will never exceed MAXLINESIZE. When a worker thread finds a match, it will prepare an item that includes the following information: the name of the file where the match has occurred, the line number of the match, and the line itself (i.e. the line string excluding the newline character at the end). If the keyword is seen, for example, in 5 separate lines, then 5 separate items will be created. If the keyword is seen more than once in the same line, then a single item will be created for that line. Whenever an item is created, then the worker thread will add this item to a memory buffer: a bounded buffer. The command-line argument passed to the server (buffersize>) specifies the size of the bounded buffer to be used by the threads. Each worker thread will add the produced items to this bounded buffer. The buffer can hold at most items. This bounded buffer can be implemented in one of two ways: 1) as a linked list of items; or 2) as a circular array of pointers to items. You can choose either one of these implementation options. A worker thread will add a new item to the end of the buffer. The buffer has to be accessed by all worker threads. While trying to insert an item, if a worker thread finds the buffer full, then the thread has to go into sleep (blocked) mode until there is space for one more item. Since all worker threads will access the buffer concurrently, the access must be coordinated. Otherwise we might end up with a corrupted buffer. You will use semaphores to protect the buffer and to synchronize the threads so that they can sleep and wake- up when necessary. Check the sem-pc.e example provided in the class website as an example on implementing these issues. Beside the worker threads, printer thread will also access the bounded buffer. The job of the printer thread will be to retrieve the items from the bounded buffer and print them into an output file. While the items are added to the buffer by the worker threads, the printer thread can work concurrently and try to retrieve the items from the buffer and print them to the output file. The printer thread will try to retrieve one item from the bounded buffer at a time. If the buffer is empty, the printer thread has to go into the sleeping mode until the buffer has at least one item. When the printer thread is successful in retrieving an item from the buffer, it will print it to the output file in the following format: 1.1 Client Program The client will issue search requests to the server. The name of the client program must be spks_client (synchronized parallel keyword search) and it will be invoked as follows: DO File Worker - . Bounded Buffer File Worker Child ./spks_client Figure 1: System Structure The output file will be named as output.txt and all child processes (only the printer thread from each child process) will write their output into this single output file. Since the output file can be modified by multiple processes concurrently, you should also synchronize the access to this output file. For this purpose, you will use the font 1 system call available in Unix providing file locking mechanism. Note that advisory locking should be sufficient for your case and you can assume that the maximum length of an output line will never exceed MAXOUTSIZE You do not need to worry about the ordering of the lines in the output file; before testing your output, we will sort the output file ourselves. Check the file_locking_solution.c example and the tutorial provided in the class website to see an example use of file locking. Pay attention to the following issues while developing your application: The output should not contain the name of the directory, just the name of the file. . Only the lines that include the exact match of an entire word is printed. If the keyword is seen more than once in a line, then a single output will be printed for that line. For example, if there was a line as "a a a". then the number of matches on this line would be 3 for the keyword "a". However, it will be reported only once. . For each search request received from the request queue, a specific bounded buffer, a specific printer thread, and bunch of worker threads (one for each file in the given directoryPath>) will be created by the child process that is assigned for that search request. . Your program will only search the first level files in the given All the worker threads for a given search request will do the same thing on different files and will fill the same bounded buffer. Your program will pass rece request to a child process and hand the next search request immediately without waiting for the created child to complete its execution. However, you should also make sure that your program does not terminate before printing all the outputs of the already received requests. To achieve this, you count the number of children created by the parent and after handling all the requests (exit request is received), then the parent process can call wait() system call in a loop for count times. We will make sure that the exit request is sent as the last request in the queue while testing your project. You need to make sure that the created child process in the server does not terminate before its worker and printer threads are done. For this purpose, first you need to join all your worker threads, then insert a dummy item to the bounded buffer to indicate the end of search, and then join your printer thread. A dummy item can be one with a negative line number, and once the printer threads receives this dummy item, it can terminate. For the thread joins to work properly. make sure your threads are set as joinable using PTHREAD_CREATE_JOINABLE. Figure 1 plots the general structure of this application: 2 Constants and Tips The is the name of the input file including all the search requests to be directed to the server. The client process will get the search requests from the input file line by line (one request at a time) and will send these requests to the server through the request message queue created by the server. The format of the search requests included in the input file will be as follows: exit 1.2 Server Program The server will accept search requests from several clients and for each request, it will search the in all the files sitting in the given using multiple threads. The server program must be named spks_server and it will be invoked as follows: MAXDIRPATH will be 1024 MAXKEYWORD will be 256 MAXLINESIZE will be 1024 MAXOUTSIZE will be 2048 Remember to use thread-safe library functions inside your threads! (for instance strtok_r() instead of strtok()). . While testing your application, remember to clean the previously created message queues be- fore restarting your application. In order to clean these system resources manually, run the clean_os.sh script provided in the class website. Some of the examples posted on the class website will be very useful for this project. ./spks_server You can make the following assumptions about the : filename:linenumber:linestring For example, if the keyword is "the" and the content of the file to be searched (assume the file name is filel.txt) is as follows: 3 Useful Clean-up Commands 1. The is a full path not exceeding the MAXDIRPATH characters 2. The is a single word not exceeding the MAXKEYWORD characters, enclosed by whitespace characters not including any whitespace characters inside. Note that a whitespace character can be either a , or a , or a character. The server process should be started before the client processes. It will immediately create a message queue that will be used to receive client requests. Similar to clients, a key will be necessary to create this message queue. Actually, clients and the server should exactly use the same key so that they can refer the same message queue. Therefore, the parameters used in ftok function should be the same in both clients and the server. However, only server should use the IPC_CREAT option in msgget function to create the message queue and wait for the client requests to arrive. Clients will not need to re-create a message queue, they will only join this message queue that is previously created by the server. Again, check the System V Message Queues examples and the tutorial provided in the class website for more information. Then, methought, the air grew denser, perfumed from an unseen censer Swung by Seraphim whose foot-falls tinkled on the rufted floor "Wretch, "I cried, "thy God hath lent thee-by these angels he hath sent thee Respite-respite and nepenthe from thy memories of Lenore; While testing your project, your server might create multiple child processes and terminate un- expectedly due to a segmentation fault. At that point, you might have several zombie processes existing in the system. In order to check current processes in the system including the "spks" name in it, you can run the following command