The Massachusetts Comprehensive Assessment System (MCAS) is a standardized exam administered

to students in the state of Massachusetts since 1993. Under law, students educated with

public funds are required to participate in statewide testing. Students take exams according to

their grade level in subjects such as Mathematics and English Language Arts; passing grades on

the Grade 10 MCAS are required for high school graduation. Exam results are used to check

student progress as well as measure school and district performance.

However, research has shown that variation in standardized test scores is often explained by factors

such as race and socioeconomic class; students of color and students from economically disadvantaged

backgrounds tend to score lower on standardized tests as a result of having less access to

educational resources than their peers. For example, a New York Times article from October 2020

reported the results of a study demonstrating that in the United States, students performed worse

on standardized tests for every additional day that was 80 degrees Fahrenheit or higherthe association

was observed for Black and Hispanic students, in addition to students with lower family

income, but not for white students.

In this problem, you will use data from the 2018 Grade 10 Mathematics MCAS to investigate evidence

of achievement gaps in standardized test score at the school level. The mcas data contains

information on 355 schools in Massachusetts. Scores on the MCAS are classified into levels: warning/

failing, needs improvement, proficient, and advanced. Students who score in the "proficient"

category are said to "demonstrate a solid understanding of challenging subject matter", while

those in the "advanced" category are said to "demonstrate a comprehensive and in-depth understanding

of rigorous subject matter"; the variable PA_perc represents the percentage of students

at a school who score at the proficient or advanced level.

The descriptions of the variables are as follows.

Variable Description

PA_perc percentage of students scoring proficient or advanced

class_size average class size

math_class_size average math class size

student_teacher_ratio student to teacher ratio

attendance_rate average percentage of days attended across students

number_of_students total number of students who took the exam

largest_minority largest minority group among the student body

white_less50 coded TRUE if less than 50% of students are white

exp_per_pupil average expenditures per pupil in USD

econ_dis percentage of economically disadvantaged students

Some additional background on certain variables:

- Whether a student is economically disadvantaged is a proxy measure for student family

income; a student is considered economically disadvantaged if they are participating in one

or more of the following programs: the Supplemental Nutrition Assistance Program (SNAP),

the Transitional Assistance for Families with Dependent Children (TAFDC), the Department

of Children and Families' (DCF) foster care program, MassHealth (Medicaid).

- Expenditures per pupil is a proxy measure for the amount of funding available to a school

district. This variable is measured by district (i.e., constant for schools in the same district).

6

Use these data to answer the following questions.

a) Explore the data.

i. Briefly summarize features of the sampled schools, focusing on the percentage of students

scoring at the proficient/advanced levels, the average expenditures per student,

and the percentage of economically disadvantaged students. Reference appropriate

numerical and graphical summaries as needed.

ii. How many schools have a student body that is greater than (or equal to) 50% white?

Of the schools where less than 50% of students are white, describe the distribution of

largest_minority.

iii. Create a single graphical summary showing the relationship between percentage of

students scoring at the proficient/advanced levels, percentage of economically disadvantaged

students, and whether a school has a student body that is less than 50% white.

Provide an informative description of what you see.

b) Examine the association between racial demographics and the percentage of economically

disadvantaged students.

i. Conduct a formal analysis comparing the percentage of economically disadvantaged

students between schools where less than 50% of the students are white and schools

where 50% or more of the students are white. Summarize the results, including reporting

and interpreting an appropriate confidence interval. Check any necessary assumptions.

ii. Among schools where less than 50% of the students are white, conduct a formal analysis

comparing the percentage of economically disadvantaged students between schools

where the largest minority group is African American versus those where the largest

minority group is Hispanic. Summarize the results and check any necessary assumptions.

c) Use a modeling approach to estimate the association between the percentage of students

scoring at the proficient/advanced levels and whether a school's student body is less than

50% white, adjusting for the following potential confounders.

i. Why it would not be advisable to include both class_size and math_class_size in a

model predicting PA_perc? Explain your answer.

ii. Would you recommend including number_of_students in a model predicting PA_perc?

Explain your answer.

iii. Fit a model predicting PA_perc from white_less50, math_class_size, attendance_rate,

and student_teacher_ratio. Interpret the slope coefficient for white_less50.

iv. Add econ_dis to the model from part iii. and interpret the slope coefficient for

white_less50. Explain the difference in the interpretation of the slope coefficient for

white_less50 in this model versus the model from part iii., using terms accessible to

someone who has not taken a statistics course.

v. Is the model from part iii. or the model from part iv. preferable for understanding

the relationship between the percentage of students scoring at the proficient/advanced

levels and whether a school's student body is less than 50% white? Explain your answer.

7

vi. Check the modeling assumptions for the model chosen in part v. Summarize the results.

d) Formally assess whether the association between the percentage of students scoring at the

proficient/advanced levels and the percentage of students who are economically disadvantaged

differs by whether less than 50% of the students are white, after adjusting for math

class size, attendance rate, and student-teacher ratio. Summarize the results.

e) Consider a school with the following features: an average math class size of 20 students,

attendance rate of 93%, and student to teacher ratio of 11, where 60% of students are African

American and 45% of students are economically disadvantaged. Suppose that in 2018, 90%

of students at this school scored at the proficient/advanced level on the Mathematics Grade

10 MCAS. Based on the model from part c) iv., is this percentage of high scorers considered

unusual for schools with the same features? Explain your answer.

f) The results from these analyses will be discussed at a future meeting of the Racial Imbalance

Advisory Council, which advises the Massachusetts Commissioner of Education and the

Board of Education on matters related to providing access to effective educational programs

for all students in the state regardless of race or socioeconomic class.

Prepare a short statement, no more than ten sentences long, summarizing the main findings

of the analyses with respect to understanding whether these data show indication of poverty

and/or race-based achievement gaps in Grade 10 Mathematics MCAS scores. Be sure to use

language that is accessible to a general audience. Reference previous numerical results as

needed.

8