The impact of school choice on student outcomes: an analysis of the Chicago Public Schools

https://doi.org/10.1016/j.jpubeco.2004.05.001Get rights and content

Abstract

We explore the impact of school choice on student outcomes in the context of open enrollment within the Chicago Public Schools (CPS). Roughly half of the students opt out of their assigned high school to attend a different CPS school, and these students are much more likely than those who remain in their assigned schools to graduate. To determine the source of this apparent benefit, we compare outcomes across (i) similar students with differential access to schooling options and (ii) travelers and non-travelers within the same school. The results suggest that, other than for students who select career academies, the observed cross-sectional benefits are likely spurious.

Introduction

The current leading elementary and secondary education reform proposals involve introducing accountability through forms of market-based competition and expanded parental choice. Proponents claim that increased choice will force the current system of local monopolies to become more responsive and efficient. Critics worry that increased choice will isolate the most disadvantaged students in the worst schools and that parents may not be informed enough to make choices in the best interests of their children.1 There is a growing body of theoretical and empirical work that presents a mixed picture of the net impact of various forms of choice.

This paper examines public school choice within the Chicago Public School system (hereafter abbreviated CPS). CPS has a number of features that make it an excellent case for analysis. First, parents have an enormous degree of flexibility in choosing which of the more than 60 CPS high schools their child will attend. For the cohorts we examine (those entering 9th grade in the Fall of 1993, 1994, or 1995), there were few restrictions on what school a child could attend within CPS. While a great deal of attention is presently focused on private school vouchers and charter schools, open enrollment is actually the most prevalent form of nontraditional parental choice. The type of intradistrict open enrollment that exists in CPS is available in more than one in every seven districts nationally, and in more than a third of large districts (NCES, 1996). Second, unlike most previous analyses of vouchers and magnet schools, which typically affect only a small percentage of students in the district, more than half of the students in CPS opt out of the neighborhood school to which they are assigned. Third, because minority and disadvantaged students are disproportionately represented in Chicago Public Schools, we are able to explore the impact of choice within an environment about which there is heightened concern.

Our empirical analysis is based on detailed student-level panel data for over 60,000 students who attend high school in the CPS. We find that students who opt out of their assigned neighborhood school are 7.6 percentage points more likely to graduate than peers who are observationally equivalent in 8th grade—off of a baseline graduation rate of 50%.2 This increment to graduation is the same order of magnitude as the gap between students at Catholic and non-Catholic schools (Evans and Schwab, 1995).

There are several competing explanations for why students who opt out of their assigned school outperform those who stay. Higher graduation rates among those who opt out may be the result of these students attending better schools or finding a school that better matches their preferences. In either of these cases, the increased graduation rates represent true benefits of open enrollment. On the other hand, the higher graduation rates among those who opt out may instead reflect negative spillovers on those who are left behind or may arise spuriously if those who opt out are better on unobserved dimensions (e.g., student motivation, parental involvement).

We begin our attempt to disentangle these competing explanations by exploiting a unique survey administered to a subset of our sample in 8th grade. The survey asks students a wide range of questions about their expectations for the future, past educational record, and parental involvement. The responses capture a number of student qualities that are generally unobservable. Even after controlling for a rich set of covariates, these variables are strongly correlated with both the likelihood of graduation and with the decision to opt out, suggesting that there are in fact systematic differences between students who opt out and those who do not.

We then compare how students with differential access to school choice fare in order to purge the analysis of the effects of unobserved student characteristics. If school choice has real effects on student outcomes, those children who live in areas with many nearby schools should on average derive the greatest benefit from the availability of school choice. Distance to nearby schools is shown to be a strong predictor of the likelihood a student will opt out of the assigned school.3 Equally important, having controlled for other factors, distance is uncorrelated with the survey responses, providing a more direct and convincing test of the exogeneity assumption than is typically possible. Empirically, we find that easy access to a career academy is associated with substantial increases in graduation likelihood, implying that career academies confer real benefits.4 In contrast, greater access to other types of schools—including high-achieving schools—is not associated with benefits on average, subject to the caveat that our estimates are relatively imprecise.

Examination of student outcomes within a given school lends further credence to the hypothesis that the gains associated with career academies are real, whereas gains for students who attend other types of schools are spurious. For most of the schools in CPS, the student body is a mix of individuals who were assigned to that school and students who elected to opt in. Presumably, these two groups experience similar peers, resources and teacher quality.5 If students who opt in are similar to those assigned to the school on unobserved dimensions, then the outcomes of these two sets of students should be similar. At career academies, the difference in graduation rates between those who opt in and those who are assigned is not statistically significant. This suggests that it is something about the school itself (e.g., the teachers, curriculum, etc.) rather than the type of students who opt in that accounts for higher graduation rates.6 In contrast, at high-achieving schools, those who opt in far outperform those assigned to the school, even after controlling for an extensive set of observable characteristics, including 8th grade test scores. This reinforces the idea that those who opt in to high-achieving schools are systematically better than observationally similar students who make other schooling choices and would outperform them regardless.

When evaluating these findings, it is important to bear in mind important weaknesses associated with the Chicago data that limit the scope of the analysis and the ability to generalize our results. The most important limitation of our data set is that we only observe students in a regime where school choice has already been introduced. Consequently, we are not able to measure the impact of school choice on a variety of outcomes that are of interest, for example, does it keep high achieving students in the public schools, does it induce families with school-age children to live in Chicago, or does the competition induced by school choice improve overall school quality. Second, our primary analysis is limited to educational attainment in high school. We have no information about college attendance or labor market outcomes, and the drop out rate is so high that sample selection poses a substantial obstacle to convincingly analyzing test scores.

The remainder of this paper is structured as follows. Section 2 describes the implementation of the school choice program in the Chicago Public School system and establishes how our study relates to the existing literature on school choice. Section 3 introduces our data set and basic patterns of student participation in the CPS choice program. Section 4 estimates the cross-sectional relationship between opting out and graduation rates, while Section 5 explores potential explanations for the observed improved educational outcomes. Section 6 concludes.

Section snippets

Previous literature

Open enrollment is only one of a variety of initiatives, including private school vouchers, charter schools, and magnet schools, that fall under the umbrella of school choice. These programs share the common goal of breaking the traditional link between neighborhood and schooling. They have the potential to increase the degree of competition faced by local public schools and, thereby, to improve school performance.7

Data description and summary statistics

Over 400,000 students are enrolled in CPS in grades K-12. As in most urban districts, students in the system are disproportionately minority (more than two-thirds) and poverty rates are well above those for the nation as a whole. Given the high rates of disadvantage and poor overall performance relative to national norms, our analysis provides evidence on the net benefits of providing choice to students with otherwise limited opportunities.

In this study, we follow three cohorts of CPS

The academic benefit associated with opting out of the assigned school

In this section, we quantify the relationship between opting out and attainment.23

Disentangling the reasons why students who opt out perform better

Having established a correlation between opting out of the assigned school and higher educational attainment, we turn our attention to trying to understand the source of those gains. We focus on three main hypotheses as to why those who opt out might do better: (1) they are better along unobservable dimensions like motivation or parental involvement, (2) they are on average traveling to “better” schools, or (3) by opting out they find a school which better matches their educational needs for

Conclusion

The Chicago Public Schools have implemented a system of school choice in which more than half of all students elect to opt out of their local assigned school to attend another public school in Chicago. Although students who opt out are more likely to graduate than observationally equivalent students who attend their neighborhood school, our evidence suggests that the leading explanation for this correlation is that those who opt out are superior along unobservable dimensions such as their

Acknowledgements

We would like to thank John Easton of the Consortium on Chicago School Research and the Chicago School Board for providing access to the Chicago Public Schools data. The National Science Foundation and Sloan Foundation provided financial support. We would also like to gratefully acknowledge Michael Greenstone, Jinyong Hahn, Caroline Hoxby, Lawrence Katz, Alan Krueger, Susanna Loeb, Charles Manski, Bruce Meyer, Kevin Murphy, Steve Rivkin, Jeff Smith, Chris Taber, three anonymous referees, the

References (28)

  • R.J. Buddin et al.

    School choice in California: who chooses private schools

    Journal of Urban Economics

    (1998)
  • D.D. Goldhaber et al.

    Testing for sample selection in the Milwaukee school choice experiment

    Economics of Education Review

    (1999)
  • R.H. Lankford et al.

    An analysis of elementary and secondary school choice

    Journal of Urban Economics

    (1995)
  • R.K. Blank

    Survey of Magnet Schools: Analyzing a Model of Quality Integrated Education

    (1983)
  • J.E. Chubb et al.

    Politics, Markets and America's Schools

    (1990)
  • J. Coleman et al.

    Parent choice and inequality

  • R.L. Crain et al.

    The Effectiveness of New York City's Career Magnet Schools: An Evaluation of Ninth-Grade Performance Using an Experimental Design

    (1992)
  • J.B. Cullen et al.

    The effect of school choice on student outcomes: evidence from randomized lotteries

    (2003)
  • T. Duax

    The Impact of Magnet Schools on a Predominantly Black Community

    (1988)
  • D. Epple et al.

    Competition between private and public schools, vouchers and peer-group effects

    American Economic Review

    (1998)
  • W. Evans et al.

    Finishing high school and starting college: do catholic schools make a difference?

    Quarterly Journal of Economics

    (1995)
  • A. Gamoran

    Student achievement in public magnet, public comprehensive, and private city high schools

    Educational Evaluation and Policy Analysis

    (1996)
  • J.P. Green et al.

    The Effectiveness of School Choice: The Milwaukee Experiment

    (1997)
  • C.M. Hoxby

    Does Competition Among Public Schools Benefit Students and Taxpayers?

    (1994)
  • Cited by (0)

    View full text