This is designed to assess your ability to apply your research design and data analysis skills toward real world problems. A essay addressing the 4 parts below.:

Research Question/Theory of Change/Evidenced Based Review/Logic Model:

This section will start with a clearly specified research question that meets the criteria of a "good question". Next, you will clearly state the program's theory of change based on the information from the case study. It is expected that you also include a logic model that aligns with the research question and theory of change.

Methods Section:

As you will read in the case study, the Program Director is concerned with 1) knowing if her program has been successful and 2) a formal program evaluation. To these ends, in the methods section you have two tasks. First, you will discuss the strengths and weaknesses related to the organization's current data collection efforts. Second, you will then provide suggestions to improve the organization's data collection moving forward as they prepare to formally evaluate their program in the future. This section will focus on all aspects of the data generating process (sampling procedures, measurements, instruments, data collection process, etc.).

Conclusion:

Your conclusion should include an overview of the project and the main findings/suggestions for the program. In this section you should also discuss any limitations related to the data analyzed and the reported findings.

TEAM READ:

IMPROVING LITERACY IN THE SEATTLE SCHOOL DISTRICT

(Note: This case study is adapted from a case published by the Electronic Hallway. Edits have been made to the original publication to better fit the needs of this course. Credit to the author and information on accessing the original publication is below)

Introduction

Merri Anderson, a spunky, 15-year-old tenth-grader from Garfield High School, waves goodbye to her classmates, who are heading to the arcade as she boards the school bus that will take her to her typical Wednesday task - a Team Read

session at Leschi Elementary School. On the bus, she takes out Polly Nguyen's Reading Log to review the work that the two have completed over the past eight months. Polly is Merri's "student reader." She is a third-grader who hates reading with a passion and consequently almost failed her reading class in second grade. Polly hardly speaks English at home since both her parents are Vietnamese immigrants, and she has problems with vocabulary. Merri remembers how Polly had problems with even basic words during their first reading. Looking at the Reading Log, which charts Polly's progress since that first session, Merri smiles with a firm sense of pride at how much Polly has improved in her reading. During a

reading session just one week before, Polly had picked a book that was a grade level above what she was used to and had managed to read it almost effortlessly. Her reading scores have improved at school, and Polly's English teacher has even approached Merri to tell her how impressed she is with the work that Merri has done.

Upon arriving at Leschi Elementary School, Merri descends from the bus and greets Tricia (Trish) McKay, Team Read Program Manager, who is standing just outside. Trish breathes in the fresh June air and looks warmly at the many high school students getting off the bus, wondering what they are thinking about their jobs as coaches in the Team Read program. She has been involved with Team Read since its inception in March 2018, helping turn the initial idea into an exciting program in ten Seattle elementary schools.

Loving her work with children and teenagers, she takes as much time as possible to visit

This case was prepared by San Cornelia Ng, research assistant at the Evans School of Public Affairs at the University of Washington, under the supervision of Marc Lindenberg, Evans School Dean and Professor. The Electronic Hallway is grateful to the agencies and individuals described for their cooperation, and especially to the Craig and Susan McCaw Foundation and its director Tricia McKay; without their assistance, the case could not have been completed.

The Electronic Hallway is administered by the University of Washington's Daniel J. Evans School of Public Affairs. This material may not be altered or copied without written permission from The Electronic Hallway. For permission, e..p@u.washington.edu,or phone (206) 616-8777. Electronic Hallway members are granted copy permission for educational purposes per the Member's Agreement (www.hallway.org).

the several project sites when the twice-a-week reading sessions are occurring. She loves watching the high school coaches file out of the bus, interact with their readers, and get absorbed in an hour-long reading session. Trish is in even deeper thought on this rainy Seattle day in June 2019. Team Read has been in full swing for more than a year. If anything, the smiling faces of these teenagers striding toward the school send an encouraging signal about the program. Moreover, she has received periodic positive feedback from teachers, site coordinators, the coaches, and even the student readers themselves. Still, she feels sure that a formal evaluation of the program is needed in the near future to measure its first year's success. She puts her thoughts about the need for an evaluation away for a moment, turns around and follows the students into the school.

THE TEAM READ PROGRAM

An Idea Blossoms and Grows

Team Read was conceived by the shared vision of a successful businessman, Craig McCaw, his wife, Susan,¹ and then-Seattle Schools Superintendent John Stanford. Craig McCaw was first inspired by Stanford and approached him with his idea. Stanford approached Joan Dore, the devoted Reading Specialist at the school district, to find ways to increase the reading skills of elementary school students. Stanford was a passionate leader who greatly believed that "every child should be a reader." Dore went to work and came up with a cross-age tutoring program using high school students as tutors, and the idea of Team Read was born in March 2018.

Team Read is an example of a private-public partnership in that private dollars and energy are working in collaboration in public education to improve literacy. Its Advising Board consists of Seattle School District Staff, Alliance for Education program representatives, Craig and Susan McCaw, and community representatives. The Team Read program is now funded through the Alliance for Education.²

The program innovators defined the purpose of Team Read as "...a Seattle School district tutoring program dedicated to increasing the reading skills of elementary students through year-long coaching by high school students.

Once the skeleton of the program started to emerge, the originators had to choose someone to head it. Given her extensive and varied work experience,³ Tricia McKay was an evident choice for Program Manager of Team Read. (Seven months later, McKay was also hired to work for the McCaw Foundation). Dore and the Seattle Public School District first identified four elementary schools to participate in the program - Dunlap, Leschi, High Point, and Rainier View. As the year progressed, another six schools were

identified - Concord, Olympic Hills, Dearborn Park, Highland Park, Madrona, and Van Asselt. Implementation proceeded in three phases: the first in March 2018, with

Team Read sessions in four schools; the second in October 2018, with an additional two schools; and the third in February 2019, with four more. At the end of the school year in June 2019, Team Read was in full operation in all ten schools, with approximately 335 students and 300 coaches. Plans to expand the programs included the addition of seven schools for the second year of implementation.

Team Read recruited its first high school tutors, or "coaches," in February of 2018. They began their participation in the program by attending the initial training session given by Joan Dore, the Seattle Schools Reading Specialist. The sessions then began the next month at the first four pilot schools.

Although things started out well, McKay did run up against some serious obstacles in getting schools to buy in to the program. She noted:

"Teachers and school administrators are very suspicious of programs like Team Read, because of their past experiences with other programs. It takes such a long time to establish and run a program, and most programs leave after a while. The schools are then left with just a program shell and lots of wasted resources. The teachers are already severely taxed within our systems, and they are just not willing to do more. I had difficulties even getting site coordinators committed to their jobs."

So McKay had to put in twice the amount of effort to enlist the stakeholders - the principals, teachers, and staff of the elementary schools. After a lengthy process of reaching out, people finally began to trust that Team Read carried good intentions for the long run. The fact that the program had the backing of John Stanford helped McKay convince people. As the year progressed, Team Read slowly settled into a steady rhythm.

The Necessary Parts That Make Up the Whole: The People Within the Program

Coaches:Team Read upholds rather stringent guidelines for the coaches. Without dedicated and talented reading coaches, the program might as well not leave the starting line. The program requires that coaches be registered at a Seattle public high school, must be a minimum of 14 years old, have a minimum 2.7 GPA and/or higher grades in reading related subjects, a good attendance record, fluency in English, and good references.

Reading coaches are also expected to be responsible and have a strong interest in working with children. In addition they are expected to:

• Be present at every tutoring session;
• Be on time and work within the established tutoring schedule outlined in training and coach manual;
• Focus on reading practice and reading exercises during the session;
• Work on a positive relationship with the reader;
• Contact the site coordinator in advance if they are late or miss a session;
• Follow directions by site coordinator, program manager, reading specialist, and other program organizers.
• Attend required training sessions.

To further develop responsibility and work ethic in coaches, they are held accountable for their behavior and actions. For example, if a coach has one unexcused absence in an academic year, he or she receives a written warning. If the coach has an additional

unexcused absence, then he or she is dismissed from the program. In addition reasons for possible dismissal include tardiness, being dishonest on forms, failing to attend training sessions, failure to follow directions, and misconduct. The process of dismissal is clearly spelled out from the start,

to instill a responsible tone in the task. It includes a verbal warning followed by a written one, a written reprimand by the program manager, and a subsequent dismissal by the program manager.

Team Read considers it necessary that coaches receive a benefit on their end. They may choose among three different options for tutoring compensation: 1) hourly pay of $8.00 (with a monthly check payment); 2) college tuition based on accumulated hourly pay plus a 50% match of that amount; or 3) school credit for 'service learning', or school-to-work, course requirements. Under option two, when a Team Read tutor registers at a college or vocational institution, the accumulated pay with the 50% match is sent directly to that institution. For example, if a student earns $1000.00 in pay, Team Read will pay $1,500.00 to the academic entity. Tuition money must be committed by one year following high school graduation.

In addition, Team Read offers all reading coaches who have tutored for at least one year the opportunity to receive a four-year scholarship of $2,500 awarded per year. One recipient of the John Stanford Scholarship is chosen every year; the runners up receive a smaller scholarship. On May 26, 2019, Craig and Susan McCaw presented the first scholarship award to an exceptional coach, Christopher Valdez from Rainer Beach High

School, who exemplified the qualities of an excellent tutor. These include effective tutoring skills, good attendance, creativity, responsibility, and cooperation and remaining on task with his student reader.

Once the coaches are selected, their preparation begins with an initial training session to give the coaches the "nuts and bolts" of tutoring a young reader. They also receive general program orientation and learn what is expected of them. During the sessions, Joan Dore, the Reading Specialist, talks with

the future coaches about the importance of their job; offers the tools and tips they need to be successful, such as comprehension and vocabulary skills; emphasizes the need to follow the structure of the sessions; and addresses how to manage difficult student reader behavior (Exhibit 3). Additional on-site

training and guidance is provided throughout the year.

Coaches are also responsible for determining if the students are improving in their reading and subsequently advancing them to higher levels. A student reader begins at a reading level predetermined in some general way by the elementary school. Team Read sites have a standard method of designating books by different reading levels using a color coding system (Exhibit 4). In this way both coaches and readers know they are moving up to more difficult books, because they are switching to a different color. The improvement the reader makes in each session is kept in a reading log which the coaches are required to fill out during the last 15 minutes of the session (Exhibit 4). The coach communicates the reader's improvement to the site coordinator, who then communicates it to the reading specialists. When the student begins reading at his grade level, he is deemed a Team Read graduate and exits the program.

Readers:Team Read attempts to reach those students who are most in need and works with the elementary schools to identify these potential participants. First, elementary schools are selected based on need and willingness to participate in the program. Then, each school identifies about 30-40 second-, third-, fourth-, and fifth-graders who are reading below grade level and who may benefit from the program. Student Intervention Teams (SIT) select students, or the site coordinator approaches teachers to ask if they notice any students with a reading problem. Eligibility criteria are based on need, as indicated by:

• Placement within the bottom quartile of district reading test scores;
• Teacher recommendations based on student classroom performance; and
• Possible retention due to failure to meet reading standards at each grade level.

Although Team Read imposes no other criteria than need, it does not serve Special Education students or those with serious discipline challenges, because it is too difficult to train high school students to be effective with special needs children, and other programs exist for that. Since Team Read is available only to a limited number of students, the selection is stringent and is based on a first-come, first-served basis. Once parents give their permission, the readers are enrolled. The site-coordinator will then match each reader to a potential coach.

Team Read students share some distinguishing characteristics. Primarily, their reading scores are well below average, and 80-90% receive "free and reduced lunch," indicating that most live in poverty. It is noticeable that there are many ethnic minorities in the program, mainly Asians and African-Americans. Furthermore, over two dozen languages are represented program-wide.

Site Coordinators:To assure that the program runs smoothly, Team Read supports a part-time site coordinator at each of the elementary schools. Usually, a teacher or other staff member of the school will assume this eight-hour-per-week position. The main task of the site coordinator is to oversee the tutoring sessions. They report to the program manager and the school principal and are evaluated informally based on their performance in carrying out their respective responsibilities. These include: conducting on-site training; completing monthly payroll; keeping necessary records; assisting with matching coaches and readers; preparing the site for the tutoring sessions; supervising transportation; modeling effective tutoring skills; and assisting in the evaluation of tutors and the program.

To prepare site coordinators to take on their role in the program, Team Read trains each one for two hours when they first begin. The coordinators and Team Read staff then get together midway through the school year for one-and-one-half hour meetings. They meet once more to recap the year.

Volunteers:As the site coordinators are generally full-time teachers or staff at their respective schools and only work part-time for Team Read, Tricia McKay saw a need for additional help at each site. Thus,

throughout the year, she formed partnerships with relevant agencies in the Seattle area. Two such partnerships are with VISTA/Americorps and the University of Washington Pipeline.

These two programs provided a total of seven full-time volunteer assistants by June 2019, one at each of seven schools. These volunteers assist the site coordinators with the Team Read program in tasks ranging from transportation logistics to working with the coaches.

The Whole: A Typical Day in the Life of Team Read

At each of the schools, Team Read runs two, one-hour sessions weekly for the entire academic year, e.g., Monday and Wednesday, or Tuesday and Thursday, depending on the school. Families are asked to commit to attending both (two) tutoring sessions a week; however, participation in the program only requires a student to attend one tutoring session per week. If a student misses more than two weeks of tutoring sessions in a semester the Site Coordinator meets with the family to remind them of their commitment, and notifies the family that 3 weeks of non-attendance in a semester results in removal from the program. Each tutoring session is standard and follows a structure listed in the Coaches' manual:

3:00PM: Reading coaches arrive at elementary school site; gather materials and snacks

3:15PM: Share snack, conversation and reading exercises

3:30PM: Begin reading session

4:15PM: Complete reading session; student readers depart

4.30PM: Reading Coaches Dismissed

To get to and from the elementary schools from their own high schools, coaches either arrange for their own transport, or opt for Team Read's transport.

Students that opt for the latter use district busses that drop them near the elementary schools for the Team Read session. At the end of the session, buses either drop students off at their normal bus stops if they bus to school or at the nearest street corner if they walk.

Once at the elementary school, coaches generally meet their readers at a spot designated by the school. For example, at Dunlap, the 40 coaches meet at the cafeteria at 3:00 pm.to gather supplies and prepare for their sessions. The site coordinator, Julie Gillett, holds a colorful flag labeled "Team Read" at the entrance of the building and reminds coaches to head to the cafeteria. At just slightly after 3:10, when the student readers are dismissed from class, she heads into the cafeteria where readers and coaches share a snack provided by Team Read. These snacks may vary from fruit to crackers and juice.

Gillett usually has quite a task managing the 15-minute snack time. "It really varies. Sometimes the cafeteria is so quiet I wonder if someone is unwell. Mostly, however, I can hardly hear myself above the constant hum of voices and laughter as I manage the group. I control the noise level and any unruly behavior, but I try to give these teenagers the room to interact with their readers in the way that they choose. This is the time for social bonding between the coaches and readers and offers a smooth transition to the reading sessions."

At 3:25, Gillett asks the group to clear their plates and get into a line of two. The site of the coaches holding the hands of the children and heading to the library warms her heart. At the library, coaches and readers head to the shelves, where the readers pick books of their choice. Coaches discourage books that are either too high or low for the reader's level.

Each pair then picks an empty spot in the library and starts the reading session. (In the case of a shortage of coaches, a coach may work with more than one student at one time.) A slow buzz soon replaces the noise of activity, and heads are bent and dotted in pairs throughout the library.

As Julie walks around to supervise the session, she comments, "There are many unforeseen things that may come up. For example, we sometimes have parents of readers coming up to complain about one thing or another just as I am trying to settle the kids, and that disrupts the rhythm of the session. Many times, coaches approach me asking for an extra chair or supplies, or they may need help with their forms. On occasion, there may be behavioral problems on the part of readers that I would intervene. Most sessions, however, are fairly uneventful."

At each school, the site coordinator is given room for creativity. Jerri Sones, from Leschi, shows a strong sense of pride as site coordinator of Team Read. She has gotten students to put up photos of Team Read sessions on the wall, with interesting captions. She has also employed a program-wide system of color-coding books according to their level of difficulty. In this way coaches and readers can keep picking books that are at their level and feel a sense of achievement as they progress from one color to another. On other occasions, site coordinators also integrate parents' day with the reading sessions, so those parents can observe the sessions to learn how to read with their children. Other innovative ideas include creating relevant games and introducing new tools for reading.

To be prepared for their own task, coaches receive training in reading skills. They also receive guidelines about reading to and with children and a coach's manual. A coach from Franklin High School feels that she has been adequately prepared to be a coach:

"During the training we learn that it's okay to be friendly. I was afraid that I had to act like a teacher. I wanted to be friends with my reader, Megan. I also learned what to do if she doesn't like to read, or refuses to read out loud for me, struggles with words or reads too slowly. For example, if she does not like to read, I know my job is to show her that reading can be fun by being nice and kind to her. And we can read the book together.

I read one line and she reads the next. We will also start laughing about a funny story or weird words. When I tell her how I used to hate reading, she feels better. And she has improved tremendously."

Coaches also run into many problems during the sessions. Eddie Bigelow, a ninth-grader from Sealth High School, loves his student Jonathon, but finds him challenging:

"Jonathon has improved. He used to have problems with certain words. Now he doesn't. He also understands more about what he's reading. It's a fun experience, but it is not easy all the time. For example, Jonathon is easily distracted and frustrated a lot. Sometimes he

takes too much time going to the bathroom or looking for a book. At times he does not understand what he's read, and he gets rude when I correct his reading or explain things.

But I know he now likes me a lot. It's just hard for him to read, which is something that he dislikes . I'd still be a coach even without pay, just to work with kids like Jonathon."

Once coaches and readers finish their session at 4:15, coaches stay on for another 15

minutes to fill out their reading logs, interact, clean up, and deal with any administrative matters. They the head out of the door at 4.30, generally feeling a sense of achievement.

FEEDBACK: A WELL-OILED MACHINE?

As the year has progressed, teachers have stopped Tricia McKay whenever she dropped by their schools to say hello. They have given her positive feedback about the program's impact on their students. They have also started to be appreciative of the fact that a successful Team Read program does in fact make their own jobs easier.

Site coordinators are also more enthusiastic about their jobs when they see how the sessions are useful. During informal feedback sessions with McKay, they share their problems as well as their hopes for future improvements. Since most site coordinators are experienced in managing children, the typical problems posed by children and teenagers do not usually perturb them. It is only when they run into issues with parents involving the program, or problems with coaches or readers that they turn to McKay. Record- keeping and payroll also sometimes pose a challenge. McKay is aware that the program has its share of problems but is able to take them in stride as part of Team Read's nature.

McKay comments on some of these problems:

"We do have dropouts, although the rate is low. Sometimes readers don't want to continue, because their parents do not see it as necessary or they exhibit serious behavioral problems that affect the other children. Coaches are generally pretty responsible, although we have some that just don't show up, show a bad attitude, or are just clearly not interested in what they are doing. Because this has a serious effect on the programs, we do dismiss them if the problem persists through the levels of warnings."

In general, McKay has a good feeling about the impact of Team Read, based on the feedback - both informal and on surveys - she has received throughout the year from principals, teachers, site-coordinators, coaches, readers, parents, and even other schools. She says, "At first it was such an uphill task even getting the schools interested in the project, because they were suspicious of it. Now we get unsolicited phone calls and mail

from other schools asking if we will be willing to launch Team Read there. This must be some indication that Team Read is doing good work."

McKay still has a nagging feeling, though, as she watches the last of the coaches file into the school bus to head home after another reading session. Although she has a general positive feeling about Team Read, she wonders if the project has been successful in increasing reading skills. Moreover, she knows that the McCaws will need more substantive proof that the program is working, and she understands that a formal evaluation is needed at this time in the project cycle.

NOTES

1. The Craig and Susan McCaw Foundation was formed in 2018. It was established for philanthropic giving on behalf of Craig and Susan McCaw. Their giving is directed to such areas as children, education, and the environment. In the international arena, the McCaws are interested in economic development and poverty alleviation. In addition, they endeavor to help give people around the world access to technology and to use technology to make the provision of services more effective. A significant portion of the gifts made from the foundation is to organizations that have an innovative and entrepreneurial approach to their mission. In 2019, the Foundation gave away $22.5 million, while in 2020, $17 million was given to both local and international projects.

1. The Alliance for Education was formed in the mid 1990's to provide private investment and involvement in the Seattle Public Schools. The McCaw Foundation directs Team Read funding through the alliance and collaborates on reading-related initiatives.
2. University of Washington, B.A. in Political Science, 1985. Legislative Assistant to U.S. Sen. Dan Evans, 1985-88. Director, Government Affairs, Washington Natural Gas 1989-95. Lecturer, Nyegezi Social Training Institute (taught political science and media history to first year journalism college students) with Jesuit Volunteers International in Tanzania, 1995-97. Team Read Program Manager, Jan. 2018 to present. Craig and Susan McCaw Fdn., July 2018 to present.

EXHIBITS

Exhibit 1. 2017-2018 Elementary School Student Outcome Data (Test Scores) (For informational purposes only. This is not the exam data)

Exhibit 2. 2018-2019 Elementary School Demographic Data (For informational purposes only. This is not the exam data)

Exhibit 3. Team Read Tutor Reading Guidelines

Exhibit 4. Team Read Reading Log

Exhibit 1. 2017-2018 Elementary School Student Outcome Data (Test Scores)

Exhibit 1. continued

Exhibit 2. 2018-2019 Elementary School Demographic Data

Exhibit 2. continued

Exhibit 3. Team Read Tutor Reading Guidelines

Exhibit 3. continued

Exhibit 3. continued

Exhibit 3. continued

Exhibit 3. continued

Exhibit 3. continued

Exhibit 3. continued

Exhibit 3F30. continued

Exhibit 4. Team Read Reading Log

Student Name:_____________ Coach Name:____________ School:________

Date	Color Code	Accuracy	Fluency	Understanding	Vocabulary	Overall reading score (add the 4 reading skill criteria)	Focus on task

Exhibit 4. Team Read Reading Log Cont.

Scoring key

Color code
1	Blue	Grade 1/semester 1	2	Blue/Black stripe	Grade 1/semester 2
3	Green	Grade 2/semester 1	4	Green/Black stripe	Grade 2/semester 2
5	Red	Grade 3/semester 1	6	Red/Black stripe	Grade 3/semester 2
7	Yellow	Grade 4/semester 1	8	Yellow/Black stripe	Grade 4/semester 2
9	Orange	Grade 5/semester 1	10	Orange/Black stripe	Grade 5/semester 2
	0=student is below 1st grade reading level and requires remedial work

Accuracy/Fluency/Understanding/Vocabulary
	1	Needs Work
	2	Good
	3	Excellent

Focus on task
	1	Student was not on task for the entire session/or behavioral issues
	2	Student was on task more 25% or less
	3	Student was on task between 26%-50% of the time
	4	Student was on task between 51%-75% of the time
	5	Student was on task between 75% or more
	6	Student was on task 100% of the time

Team Read Case Part 2Data Collection

You receive a call from Trish McKay, Team Read Program Manager. Her organization is seeking help evaluating the Team Read program, and she request to meet with you to discuss contracting you to conduct the evaluation. The contract involves two tasks. First, she would like you to examine their current data collection practices and make recommendations to ensure the organization will have appropriate data in the future to conduct a formal evaluation. The second task involves reviewing existing data the organization has collected to understand if the program is working. A few excerpts from your conversation with Trish McKay and your notes from the meeting are below.

In your meeting, Trish McKay admits that she did not do the best job collecting information needed to evaluate the Team Read program. She stated,

At the start, I was so focused on getting the program off the ground that I didn't think much about collecting data. After all, most of the schools were so reluctant to let the Team Read program in that I spent nearly all of my time building rapport and convincing school administration and teachers that we are here for the long -haul. There was no point in thinking about collecting data if there was no program to evaluate!

I lucked out to some extent. An AmeriCorps volunteer with the Team Read program, who was finishing her Masters degree, requested to evaluate our program as part of a class project. I was excited because I knew we needed to collect information to evaluate if the Team Read Program was working, and this AmeriCorps volunteer shows up and takes this task off my already over-filled plate. The very first action she took was to update our reading logs. She included additional information that could be used to assess the progress of our program. Before she updated our reading logs, they were simply attendance sheets. (See Exhibit 4 to view the updated reading log).

The AmeriCorps Volunteer selected a few sites to collect data from. There wasn't anything special about these sites. They were selected because the site was either convenient for her or she was friendly with the Site Coordinator.

Watching her collect and enter all of that information really got me thinking about how we need to improve our data collection processes. It took her hours just to enter a small amount of information. We are already strapped for time. There are several hundred kids in this program, and we are growing by leaps and bounds. I'm worried that our current pen-and-paper methods will be a nightmare when it comes time to formally evaluate the program. We have to figure out a better way to know if we are meeting our goals. To be honest, I haven't even looked at the data she gave me. She did say that she made notes in the dataset on how she calculated everything. I hope it is enough for us to do something with, at least for now. I had a conversation with my Board last month, and we all agree that is imperative that we, as a group, decide how best to evaluate our programs.

As you spend more time talking to Tricia McKay you pick up some additional details. Below are excerpts from your notes.

1. There are data on only 2 of the 4 initial schools that started the program in March 2018. There are also data on 1 school that started the program in October 2018 and 1 school that started the program in February 2019. The data includes every child at the school that participated in the program, even if the student dropped out or entered the program late.
2. Upon entering the program, students take a short reading test to determine the most appropriate reading level of each student. Ms. McKay really likes this instrument because it's computer based and only takes about 20 minutes to administer. This assessment was developed in collaboration with several reading specialist in the school district. As a side note, it has gained a lot of popularity, and several school reading specialists in the school district asked Ms. McKay if they could use the instrument for their own purposes.
3. All other reading progress data is collected by the Reading Coaches. See Exhibit 4. Reading Coaches receive training on how to identify reading growth at the beginning of the school year, but do not receive any follow up training. The Site Coordinators do not provide oversite of the information entered into reading logs, but are available if a Reading Coach need assistance. Reading logs are turned into the Site Coordinators at the end of each month, and then submit the logs to Tricia McKay.
4. The data set constructed by the AmeriCorps Volunteer includes the following information:
   1. Color coded reading level. There are two data points1) beginning reading level. This measure is taken at the beginning of the year OR when the student joins the program and 2) the reading level at the end of the academic year.
   2. Reading Accuracy. Measured as the modal score.
   3. Reading Fluency. Measured as the modal score.
   4. Reading Understanding. Measured as the modal score.
   5. Reading Vocabulary. Measured as the modal score.
   6. Overall reading score. Measured as the sum of items b-e
   7. Focus on task. Measured as the average focus score throughout the academic year
   8. Attendance. Measured as the number of tutoring sessions attended.
   9. Student demographic data

****Notes for the analyzing the data. 1) you can proceed as if the data meets necessary assumptions. 2) Much of the data is ordinal. If you choose to treat the data as continuous, you should include a justification for your decision.