Over the last decade, the District of Columbia school system has seen a plethora of major educational changes, from open enrollment and a burgeoning charter sector to overhauled teacher evaluation and pay systems.
A new Mathematica study suggests no single change, but all of them together, spurred public schools in the nation鈥檚 capital to improve faster than states and county districts that didn鈥檛 use so-called market-based education reforms since the 1990s.
And the educational changes contributed more to the increase in achievement than any gentrification or changes in the student population, the researchers said.
鈥淲e really focus on the overall picture rather than trying to break it into individual components,鈥� said Duncan Chaplin, senior researcher at Mathematica, who co-authored the study with researchers Dallas Dotter and Maria Bartlett. 鈥淭he improvements in grade 4 were very impressive, and they didn鈥檛 fade out by grade 8. ... [The benefits] didn鈥檛 fade out as kids got older, and they didn鈥檛 fade out over years鈥� as the changes were implemented.
Chaplin said the study provides evidence in favor of market-based education reforms that have gained traction in many urban districts. However, the results also highlight potential pitfalls for less high-profile districts attempting to copy the capital city鈥檚 formula. The study did not, however, examine some other reforms, such as the common core, that were overlapped with the market-based changes.
Washington D.C.'s major education changes started with the passage of the Public Education Reform Amendment Act in 2007, which abolished the district鈥檚 board of education and created a mayor-appointed chancellor to run the schools. Then-Mayor Adrian Fenty and the first Chancellor Michelle Rhee launched aggressive and controversial structural changes, including implementing a high-stakes teacher evaluation system in 2009. The district gave the public charter school board control over approvals and renewals based on performance; the portion of students attending charter schools in the district has risen from 27 percent in 2007 to 46 percent a decade later. Finally, in 2014, the district opened a districtwide enrollment system for students.
The Mathematica study spans close to 30 years: five waves of 4th and 8th graders before reforms began in 2007 and three waves of students afterward, from 1996 to 2017 for math and 1998 to 2017 in reading. Researchers used expanded data from the National Assessment of Educational Progress in math and reading in both grades to compare the achievement of students in Washington, D.C., to that of two separate groups鈥攐ne of states and one of counties鈥攚hich participated in NAEP during that time. The researchers left out states or counties which had implemented similar systemic education changes to the District of Columbia鈥檚: abolishing teacher tenure, increasing charter school enrollment, and using unified school enrollment systems.
A look at the numbers
During that time, the District of Columbia also saw an influx of more white and higher-income students. Since 2007, the share of Black students has fallen 15 percentage points and the share of white students has risen by the same amount, and the NAEP has changed the students it tests to reflect this. However, after controlling for those demographic changes, the researchers found Washington, D.C., students鈥攁nd Black students in particular鈥攎ade much faster academic progress compared to states and counties that did not implement the same collection of education changes. After controlling for demographics, public students in the nation鈥檚 capital performed on average 12 percentile points鈥攁bout a third of a standard deviation鈥攈igher in math in grades 4 and 8 and more than a quarter of a standard deviation higher in reading in grade 4. The reading benefits faded by grade 8, however.
That meant Black students in District of Columbia schools scored on average 8.3 points higher in math at 8th grade and 9.5 points higher in 4th grade, while Hispanic students scored on average 9.5 points higher in math at 8th grade and 8 points higher in 4th grade. White students in Washington performed more than 6 points better on average in 4th grade math. It鈥檚 not clear why white students showed less benefit, but Chaplin noted that in highly racially segregated schools in the district, white students may have been more likely to have started out in higher-performing schools.
Evidence of Washington鈥檚 rapid improvement isn鈥檛 new. Earlier this year, the Council of the Great City Schools judged it the fastest-improving of all large urban school districts over the last decade. The district outperformed the CGCS鈥檚 expectations by a greater amount from 2009-19, and in more subject-and-grade combinations, than any other large urban district.
The Mathematica study does not analyze what specific elements may have driven the District of Columbia鈥檚 improvements, and does not compare the city鈥檚 students to other districts that used similar market-oriented education changes, but its results are in line with those seen in Detroit, Mich., and in New Orleans following Hurricane Katrina, when both urban districts boosted charter schools and school choice, and overhauled teacher tenure.
Douglas Harris, an economics professor at Tulane University and the author of 鈥�,鈥� who has studied New Orleans鈥� education changes post-Katrina, suggested the different policies may help build on each other over time to eliminate the lowest-performing schools or teachers, leading the improvements to show up after a few years.
For example, 鈥渁t least in new Orleans, it was really driven by the process of taking over low-performing schools. ... Basically, they would start some [charters], some of them would work and were effective and did well, and those stayed open, and the ones that didn鈥檛 succeed, they gave over to another charter operator and let them try,鈥� Harris said. 鈥淚n the process, they kept winnowing out the low-performing schools and that lifted up the average over the first seven or eight years.鈥�
However, Harris cautioned that both New Orleans and Washington have been high-profile, nationally watched turnaround districts in culturally rich cities, able to draw both foundation support and teacher and leader talent to support systemic overhauls to schools and districts. Smaller and less well-known cities and rural areas may have difficulty replicating and sustaining large-scale reforms.
鈥淚f it鈥檚 the nation鈥檚 capital, that鈥檚 also kind of an attractive place to be for the younger folks who tend to be involved in that kind of school reform. So that gives them an advantage in attracting people,鈥� Harris said. 鈥淪o I think that context is also important, and it also is relevant to the rural-urban piece鈥� of implementing market-based education changes.
Harris said he expects to release a separate study comparing the implementation and progress of different districts, like New Orleans and Washington D.C., that implemented market-based reforms, in the next few months.
