Problem Set 4

In this problem set you will use the skills you have developed to critically assess an empirical study. To complete the problem set, you will need the following two readings:

Roland G. Fryer & Steven D. Levitt, 2004."Understanding the Black-White Test Score Gap in the First Two Years of School," The Review of Economics and Statistics, MIT Press, vol. 86(2), pages 447-464.

[This article is available in courseworks]

Freakonomics: A Rogue Economist Explores the Hidden Side of Everything: Steven Levitt and Stephen Dubner, 2005. Pages 161-176.

[Copies of this text will be distributed in class.]

The policy issue underpinning this study is the black-white test score gap.As a policy maker, it is critical not only to know that this gap exists but to assess the underlying reasons for its existence.As reported by Education Next:

On average, black students typically score one standard deviation below white students on standardized testsroughly the difference in performance between the average 4th grader and the average 8th grader. Historically, what has come to be known as the black-white test-score gap has emerged before children enter kindergarten and has tended to widen over time. What are the causes of this persistent gap in achievement? In study after study, scholars have investigated the effects of differences among white and black students in their socioeconomic status, family structure, and neighborhood characteristics and in the quality of their schools. To be sure, socioeconomic status and the trappings of poverty are important factors in explaining racial differences in educational achievement. Yet a substantial gap remains even after these crucial influences are accounted for.

Part I - Interpreting the regression results of the study

Read the Fryer and Levitt article, and respond to the following questions:

1. Interpret the coefficient on Black in regression (1) of Table 2
2. Explain why the coefficient on Black changes from regression (1) to regression (2) of Table 2. Explain the direction of the change.
3. Based on Table 2, what is the average difference in math test scores between Hispanics and Blacks? How would you test whether this difference is statistically significant?
4. As reported in section V, the gap in test scores between black and white students increases as children age (from Fall Kindergarten to Spring First grade). Section VI advances six possible hypotheses behind this result. Hypothesis 3 focuses on summer setbacks as a possible explanation. The authors indicate (p.458) that "...black students lose slightly relative to whites over the summer in math (the gap rises from -0.097 to -0.134), but the null hypothesis of no change cannot be rejected." Explain in what way this statement supports the authors' conclusion that hypothesis 3 is not supported by the evidence.
5. On p. 169 of the Freakonomics piece, Levitt and Dubner argue that two factors that "mattered" were "the child's mother was thirty or older at the time of her first child's birth" and "the child had low birthweight". Explain what "mattering" means in this context, and indicate where (i.e. in which page/table of the Fryer and Levitt article) is the supporting evidence for these claims.

Part II - Critically assessing the study

1. In page 448 of Fryer and Levitt, the authors indicate "There are three leading explanations for why our results differ so sharply from earlier research such as Phillips et al. (1998a): (1) nonrandom sampling in the data sets used in earlier studies, (2) real gains by recent cohorts of blacks, and (3) better covariates in ECLS." Explain how each of these factors is a threat to the validity of the previous studies referenced. In doing so, explain what type of validity (internal or external) is being threatened by each factor.
2. List two of the authors' conclusions for which you think they provide strong supporting evidence. Describe the evidence and explain why you find it compelling.
3. List two of the authors' conclusions for which you think they do not provide strong supporting evidence. Describe the evidence and explain why you do not find it compelling.

(continues to Part III) Part III - Taking the findings to the policymakers

In this section, assume you work in the division of policy analysis at the US Department of Health and Human Services. You are asked to assist in the preparation of a memo that will be used to brief a Congressional committee on the black-white test score gap.

1. At the meeting, a congressional aide looks at table 2 and notices the coefficient on the variable "WIC participant" [1]. Struggling to recall what he heard in his econometrics class, he exclaims "the coefficient on WIC participant is negative and statistically significant in all the regressions. This is a clear indication that the WIC program does not work!" How would you respond, in simple language that could be understood by others at the meeting? An example of what "simple language" means for this question can be found in p. 172-174 of the Freakonomics piece.

[1] "The Special Supplemental Nutrition Program for Women, Infants, and Children - better known as the WIC Program - serves to safeguard the health of low-income women, infants, & children up to age 5 who are at nutritional riskby providing nutritious foods to supplement diets,information on healthy eating, and referrals to health care."