Programming Assignment 1 description (tentative) CIS 545 Architecture and Operating Systems Due: February 15 and March 10 @11:59 pm, 2022 1. Preparation Before beginning your work, please read the following carefully ? Chapters 3-6 from Silberschatz (9t h Edition) ? Lecture slides on Processes, Threads, CPU Scheduling and Process Synchronization ? Pintos Introduction ? Pintos Reference Guide (Sections referred below) ? Complete Pintos Documentation (PDF file) -- for reference only 2. Task: Implement the Threading Component of Pintos OS In this project, your team is asked to perform kernel level programming of the Threading component in the Pintos operating system. This project will help you to better understand threading, synchronization, and scheduling concepts. Pintos is a simple operating system framework for the 80x86 architecture developed at Stanford University. It is considered as a successor of the Nachos instructional OS from UC-Berkeley. You will be given a functional OS with minimally functional thread system. Your job is to extend the functionality of this system to gain a better understanding of synchronization problems. Before beginning this assignment, make sure you read these sections from Pintos Reference Guide: section A.1 Pintos Loading, A.2 Threads, A.3 Synchronization, and Appendix B (4.4BSD In VirtualBox software, from the File menu, choose "Import Appliance" and point to the downloaded "pintos.ova". You do not need to change any other settings for this procedure. Choose UB-Pintos from the left pane, and click on "Start", your virtual machine should now boot. Here is the login information if needed: Username: os-class Password: os-class To learn how to run, debug and test Pintos code, please read the Pintos Introduction. 4. Implementation of the Project You will be working primarily in the threads directory of the source tree for this assignment, with some work in the devices directory on the side. Compilation should be done in the threads directory. 4.1 Understanding Threads The first step is to read and understand the code for the initial thread system. Pintos already implements thread creation and thread completion, a simple scheduler to switch between threads, and synchronization primitives (semaphores, locks, condition variables, and optimization barriers). Some of this code might seem slightly mysterious. If you haven't already compiled and run the base system, as described in the introduction (see section 1. Introduction), you should do so now. You can read through parts of the source code to see what's going on. If you like, you can add calls to printf() almost anywhere, then recompile and run to see what happens and in what order. You can also run the kernel in a debugger and set breakpoints at interesting spots, single-step through code and examine data, and so on. When a thread is created, you are creating a new context to be scheduled. You provide a function to be run in this context as an argument to thread_create(). The first time the thread is scheduled and runs, it starts from the beginning of that function and executes in that context. When the function returns, the thread terminates. Each thread, therefore, acts like a mini-program running inside Pintos, with the function passed to thread_create() acting like main(). At any given time, exactly one thread runs and the rest, if any, become inactive. The scheduler decides which thread to run next. (If no thread is ready to run at any given time, then the special "idle" thread, implemented in idle(), runs.) Synchronization primitives can force context switches when one thread needs to wait for another thread to do something. The mechanics of a context switch are in threads/switch.S, which is 80x86 assembly code. (You don't have to understand it.) It saves the state of the currently running thread and restores the state of the thread we're switching to. Using the GDB debugger, slowly trace through a context switch to see what happens (see section E.5 GDB). You can set a breakpoint on schedule() to start out, and then single-step from there. Be sure to keep track of each thread's address and state, and what procedures are on the call stack for each thread. You will notice that when one thread calls switch_threads(), another thread starts running, and the first thing the new thread does is to return from switch_threads(). You will understand the thread system once you understand why and how the switch_threads() that gets called is different from the switch_threads() that returns. See section A.2.3 Thread Switching, for more information. Warning: In Pintos, each thread is assigned a small, fixed-size execution stack just under 4 kB in size. The kernel tries to detect stack overflow, but it cannot do so perfectly. You may cause bizarre problems, such as mysterious kernel panics, if you declare large data structures as non-static local variables, e.g. int buf[1000];. Alternatives to stack allocation include the page allocator and the block allocator (see section A.5 Memory Allocation). 4.2 Source Files For a brief overview of the files in the threads/ directory, please see Section 2.1.2 Source Files in the Pintos Reference Guide. You will not need to modify most of this code, but the hope is that overviewing this section will give you a start on what code to look at. Also, please take a quick look at the files in the devices/ and lib/ directories of the pintos source tree. 4.3 Synchronization Proper synchronization is an important part of the solutions to these problems. Any synchronization problem can be easily solved by turning interrupts off: while interrupts are off, there is no concurrency, so there's no possibility for race conditions. Therefore, it's tempting to solve all synchronization problems this way, but don't. Instead, use semaphores, locks, and condition variables to solve the bulk of your synchronization problems. Read the tour section on synchronization (see section A.3 Synchronization) or the comments in threads/synch.c if you're unsure what synchronization primitives may be used in what situations. Any implementation turning the interrupts off for synchronization purposes, will not get full credits! In the Pintos projects, the only class of problem best solved by disabling interrupts is coordinating data shared between a kernel thread and an interrupt handler. Because interrupt handlers can't sleep, they can't acquire locks. This means that data shared between kernel threads and an interrupt handler must be protected within a kernel thread by turning off interrupts. This project only requires accessing a little bit of thread state from interrupt handlers. For the alarm clock, the timer interrupt needs to wake up sleeping threads. In the advanced scheduler, the timer interrupt needs to access a few global and per-thread variables. When you access these variables from kernel threads, you will need to disable interrupts to prevent the timer interrupt from interfering. When you do turn off interrupts, take care to do so for the least amount of code possible, or you can end up losing important things such as timer ticks or input events. Turning off interrupts also increases the interrupt handling latency, which can make a machine feel sluggish if taken too far. The synchronization primitives themselves in synch.c are implemented by disabling interrupts. You may need to increase the amount of code that runs with interrupts disabled here, but you should still try to keep it to a minimum. Disabling interrupts can be useful for debugging, if you want to make sure that a section of code is not interrupted. You should remove debugging code before turning in your project. (Don't just comment it out, because that can make the code difficult to read.) There should be no busy waiting in your submission. A tight loop that calls thread_yield() is one form of busy waiting. 4.4 Alarm Clock Reimplement timer_sleep(), defined in devices/timer.c. Although a working implementation is provided, it "busy waits," that is, it spins in a loop checking the current time and calling thread_yield() until enough time has gone by. Reimplement it to avoid busy waiting. Any implementation using busy-waiting will not get full credits! Function: void timer_sleep (int64_t ticks) -- Suspends execution of the calling thread until time has advanced by at least x timer ticks. Unless the system is otherwise idle, the thread need not wake up after exactly x ticks. Just put it on the ready queue after they have waited for the right amount of time. timer_sleep() is useful for threads that operate in real-time, e.g. for blinking the cursor once per second. The argument to timer_sleep() is expressed in timer ticks, not in milliseconds or any another unit. There are TIMER_FREQ timer ticks per second, where TIMER_FREQ is a macro defined in devices/timer.h. The default value is 100. We don't recommend changing this value, because any change is likely to cause many of the tests to fail. Separate functions timer_msleep(), timer_usleep(), and timer_nsleep() do exist for sleeping a specific number of milliseconds, microseconds, or nanoseconds, respectively, but these will call timer_sleep() automatically when necessary. You do not need to modify them.If your delays seem too short or too long, reread the explanation of the -r option to pintos (see section 1.1.4 Debugging versus Testing). 4.5 Priority Scheduler Implement priority scheduling in Pintos. When a thread is added to the ready list that has a higher priority than the currently running thread, the current thread should immediately yield the processor to the new thread. Similarly, when threads are waiting for a lock, semaphore, or condition variable, the highest priority waiting thread should be awakened first. A thread may raise or lower its own priority at any time, but lowering its priority such that it no longer has the highest priority must cause it to immediately yield the CPU. Thread priorities range from PRI_MIN (0) to PRI_MAX (63). Lower numbers correspond to lower priorities, so that priority 0 is the lowest priority and priority 63 is the highest. The initial thread priority is passed as an argument to thread_create(). If there's no reason to choose another priority, use PRI_DEFAULT (31). The PRI_ macros are defined in threads/thread.h, and you should not change their values. One issue with priority scheduling is "priority inversion". Consider high, medium, and low priority threads H, M, and L, respectively. If H needs to wait for L (for instance, for a lock held by L) , and M is on the ready list, then H will never get the CPU because the low priority thread will not get any CPU time. A partial fix for this problem is for H to "donate" its priority to L while L is holding the lock, then recall the donation once L releases (and thus H acquires) the lock. Implement priority donation. You will need to account for all different situations in which priority donation is required. Be sure to handle multiple donations, in which multiple priorities are donated to a single thread. You must also handle nested donation: if H is waiting on a lock that M holds and M is waiting on a lock that L holds, then both M and L should be boosted to H's priority. If necessary, you may impose a reasonable limit on depth of nested priority donation, such as 8 levels. You must implement priority donation for locks. You need not implement priority donation for the other Pintos synchronization constructs. You do need to implement priority scheduling in all cases. Finally, implement the following functions that allow a thread to examine and modify its own priority. Skeletons for these functions are provided in threads/thread.c. Function: void thread_set_priority (int new_priority) -- Sets the current thread's priority to new_priority. If the current thread no longer has the highest priority, yields. Function: int thread_get_priority (void) -- Returns the current thread's priority. In the presence of priority donation, returns the higher (donated) priority. You need not provide any interface to allow a thread to directly modify other threads' priorities. 4.6 Multilevel Feedback Queue Scheduler Implement a multilevel feedback queue scheduler similar to the 4.4BSD scheduler to reduce the average response time for running jobs on your system. See section B. 4.4BSD Scheduler , for detailed requirements. Like the priority scheduler, the advanced scheduler chooses the thread to run based on priorities. However, the advanced scheduler does not do priority donation. Thus, we recommend that you have the priority scheduler working, except possibly for priority donation, before you start work on the advanced scheduler. You must write your code to allow us to choose a scheduling algorithm policy at Pintos startup time. By default, the priority scheduler must be active, but we must be able to choose the 4.4BSD scheduler with the -mlfqs kernel option. Passing this option sets thread_mlfqs, declared in threads/thread.h, to true when the options are parsed by parse_options(), which happens early in main(). When the 4.4BSD scheduler is enabled, threads no longer directly control their own priorities. The priority argument to thread_create() should be ignored, as well as any calls to thread_set_priority(), and thread_get_priority() should return the thread's current priority as set by the scheduler. 5. Testing Your project grade will be based on the tests in the pintos/src/tests/threads directory. Each project has several tests, each of which has a name beginning with tests. To completely test your submission, invoke make check from the project build directory. This will build and run each test and print a "pass" or "fail" message for each one. When a test fails, make check also prints some details of the reason for failure. After running all the tests, make check also prints a summary of the test results. You can also run individual tests one at a time. A given test t writes its output to t. output, then a script scores the output as "pass" or "fail" and writes the verdict to t.result. To run and grade a single test, make the .result file explicitly from the build directory, e.g. make tests/threads/alarm-multiple.result If make says that the test result is up-to-date, but you want to re-run it anyway, either run make clean or delete the .output file by hand. By default, each test provides feedback only at completion, not during its run. If you prefer, you can observe the progress of each test by specifying VERBOSE=1 on the make command line. You can also invoke different options in pintos e.g. make check PINTOSOPTS='-j 1' to select a jitter value of 1 (see section 1.1.4 Debugging versus Testing). 6. Design Document A copy of the Project 1 Design Document (threads.tmpl) can be found and also inside pintos/doc/. Copy the threads.tmpl file to threads.txt for your submission. Leave the header as it is. Change the FirstName LastName into your corresponding team members details. +--------------------+ | CIS 545 | | PROJECT 1: THREADS | | DESIGN DOCUMENT | +--------------------+ ---- GROUP ---- >> Fill in the names and email addresses of your group members. FirstName LastName FirstName LastName FirstName LastName ---- PRELIMINARIES ---- >> If you have any preliminary comments on your submission, notes for the >> TAs, or extra credit, please give them here. >> Please cite any offline or online sources you consulted while >> preparing your submission, other than the Pintos documentation, course >> text, lecture notes, and course staff. ALARM CLOCK =========== ---- DATA STRUCTURES ---- >> A1: Copy here the declaration of each new or changed `struct' or >> `struct' member, global or static variable, `typedef', or >> enumeration. Identify the purpose of each in 25 words or less. ---- ALGORITHMS ---- >> A2: Briefly describe what happens in a call to timer_sleep(), >> including the effects of the timer interrupt handler. >> A3: What steps are taken to minimize the amount of time spent in >> the timer interrupt handler? ---- SYNCHRONIZATION ---- >> A4: How are race conditions avoided when multiple threads call >> timer_sleep() simultaneously? >> A5: How are race conditions avoided when a timer interrupt occurs >> during a call to timer_sleep()? ---- RATIONALE ---- >> A6: Why did you choose this design? In what ways is it superior to >> another design you considered? PRIORITY SCHEDULING =================== ---- DATA STRUCTURES ---- >> B1: Copy here the declaration of each new or changed `struct' or >> `struct' member, global or static variable, `typedef', or >> enumeration. Identify the purpose of each in 25 words or less. >> B2: Explain the data structure used to track priority donation. >> Use ASCII art to diagram a nested donation. (Alternately, submit a >> .png file.) ---- ALGORITHMS ---- >> B3: How do you ensure that the highest priority thread waiting for >> a lock, semaphore, or condition variable wakes up first? >> B4: Describe the sequence of events when a call to lock_acquire() >> causes a priority donation. How is nested donation handled? >> B5: Describe the sequence of events when lock_release() is called >> on a lock that a higher-priority thread is waiting for. ---- SYNCHRONIZATION ---- >> B6: Describe a potential race in thread_set_priority() and explain >> how your implementation avoids it. Can you use a lock to avoid >> this race? ---- RATIONALE ---- >> B7: Why did you choose this design? In what ways is it superior to >> another design you considered? ADVANCED SCHEDULER ================== ---- DATA STRUCTURES ---- >> C1: Copy here the declaration of each new or changed `struct' or >> `struct' member, global or static variable, `typedef', or >> enumeration. Identify the purpose of each in 25 words or less. ---- ALGORITHMS ---- >> C2: Suppose threads A, B, and C have nice values 0, 1, and 2. Each >> has a recent_cpu value of 0. Fill in the table below showing the >> scheduling decision and the priority and recent_cpu values for each >> thread after each given number of timer ticks: timer recent_cpu priority thread ticks A B C A B C to run ----- -- -- -- -- -- -- ------ 0 4 8 12 16 20 24 28 32 36 >> C3: Did any ambiguities in the scheduler specification make values >> in the table uncertain? If so, what rule did you use to resolve >> them? Does this match the behavior of your scheduler? >> C4: How is the way you divided the cost of scheduling between code >> inside and outside interrupt context likely to affect performance? ---- RATIONALE ---- >> C5: Briefly critique your design, pointing out advantages and >> disadvantages in your design choices. If you were to have extra >> time to work on this part of the project, how might you choose to >> refine or improve your design? >> C6: The assignment explains arithmetic for fixed-point math in >> detail, but it leaves it open to you to implement it. Why did you >> decide to implement it the way you did? If you created an >> abstraction layer for fixed-point math, that is, an abstract data >> type and/or a set of functions or macros to manipulate fixed-point >> numbers, why did you do so? If not, why not? SURVEY QUESTIONS ================ Answering these questions is optional, but it will help us improve the course in future quarters. Feel free to tell us anything you want--these questions are just to spur your thoughts. You may also choose to respond anonymously in the course evaluations at the end of the quarter. >> In your opinion, was this assignment, or any one of the three problems >> in it, too easy or too hard? Did it take too long or too little time? >> Did you find that working on a particular part of the assignment gave >> you greater insight into some aspect of OS design? >> Is there some particular fact or hint we should give students in >> future quarters to help them solve the problems? Conversely, did you >> find any of our guidance to be misleading? >> Do you have any suggestions for the TAs to more effectively assist >> students, either for future quarters or the remaining projects? >> Any other comments? We recommend that you read the design document template before you start working on the project. See section D. Project Documentation, for a sample design document that goes along with a fictitious project. You will need to decide and describe the main data structures, algorithms, and synchronization mechanisms that you are using / planning to use for each component of the project. 7. Grading The grading of the project will be done according to the following rubric : ? (18 points) A completely working Alarm Clock implementation that passes all six (6) tests. ? (38 points) A fully functional Priority Scheduler that passes all twelve (12) tests. ? (37 points) A working advanced scheduler that passes all nine (9) tests. Run make check and make grade to see how many total points you receives from implementation (out of 93) and what is the grade. Functionality and robustness of alarm clock (tests/threads/Rubric.alarm): 4 / 4 tests/threads/alarm-single 4 / 4 tests/threads/alarm-multiple 4 / 4 tests/threads/alarm-simultaneous 4 / 4 tests/threads/alarm-priority 1 / 1 tests/threads/alarm-zero 1 / 1 tests/threads/alarm-negative Functionality of priority scheduler (tests/threads/Rubric.priority): 3/ 3 tests/threads/priority-change 3 / 3 tests/threads/priority-preempt 3 / 3 tests/threads/priority-fifo 3 / 3 tests/threads/priority-sema 3 / 3 tests/threads/priority-condvar 3 / 3 tests/threads/priority-donate-one 3 / 3 tests/threads/priority-donate-multiple 3 / 3 tests/threads/priority-donate-multiple2 3 / 3 tests/threads/priority-donate-nest 5 / 5 tests/threads/priority-donate-chain 3 / 3 tests/threads/priority-donate-sema 3 / 3 tests/threads/priority-donate-lower Functionality of advanced scheduler (tests/threads/Rubric.mlfqs): 5 / 5 tests/threads/mlfqs-load-1 5 / 5 tests/threads/mlfqs-load-60 3 / 3 tests/threads/mlfqs-load-avg 5 / 5 tests/threads/mlfqsrecent-1 5 / 5 tests/threads/mlfqs-fair-2 3 / 3 tests/threads/mlfqs-fair-20 4 / 4 tests/threads/mlfqs-nice-2 2 / 2 tests/threads/mlfqs-nice-10 5 / 5 tests/threads/mlfqs-block ? CHECK CODE: Alarm clock implementation should not be implemented by using busywaiting ? CHECK CODE: Interrupts should not be turned off as the synchronization mechanism ? The source code score break down Alarm Clock implementation - 20% Priority Scheduler - 40% MLFQ Scheduler - 40% ? This score is scaled down from 100% to 90% to be your source code score. ? Your design document is the remaining 10% of your Project-1 grade. ? Check autograder submission score to be consistent with what you get on your VM. ? Youll have unlimited submission, submit early and re-submit. 8. What to Submit? 1. You need to write a design document for your project as described in section 6. A soft copy of this design document (as text file) should be submitted before the source code deadline. The design document is due February 15 t h @11:59 pm. (NO LATE SUBMISSION) 2. You need to submit the complete source tree (all source files) of your project. The whole package should compile when the tester simply types make in the source code directory. The project source code submission is due March 10t h@11:59 pm. (NO LATE SUBMISSION) REFERENCES ? Pintos Reference Manual

Attachments:

pintos-pa1-de....pdf