The question of charter school performance is vital. However this line of critique is largely irrelevant. The overwhelming majority of education policy and practice is not national, but local — charter results in Dayton and Detroit have little to do with school decisions in Denver. And in Denver, the very same CREDO study explicitly stated — and further analysis of more recent performance data confirms — that charter schools are doing far better than their traditional school peers.

Indeed, school districts across Colorado would be well advised to look at Denver’s model with an eye to replicating its success.

It’s helpful to quickly revisit the essentials. A central premise of charter schools is simple: encourage innovation and a variety of school models. Measure outcomes. Expand the good schools, and change or close the bad ones. This basic combination of innovation, evaluation, and adjustment should lead — particularly over time — to more high-quality schools and better outcomes for students.

Denver is a vibrant example of this theory.  Over the past several years a consistent (if fragile) coalition on the Board of Education has established a solid process for encouraging and approving innovative charter proposals. Denver Public Schools (DPS) created a comprehensive annual evaluation system to measure school quality. And both forged the collective political will to close charters that do poorly.

Denver’s charters now display numerous models, including Expeditionary Learning, dual-language immersion, and entrepreneurship. These innovations have indeed produced a wide variation in quality: on the 2010 School Performance Framework, three of the top five schools were charters – and so were two of the bottom five.

However the best charter schools are expanding to serve more students, while the worst are being reconfigured and face closure. And in the aggregate, charter schools in Denver are now doing far better than their traditional peers on both quantitative academic criteria and qualitative metrics.

In Denver, we have a rare and somewhat unique ability to make comparisons based on two local frameworks for measuring school quality: the Colorado Growth Model (which measures the academic growth in individual students from year to year), and DPS’s School Performance Framework (which derives academic data from the growth model but also includes non-academic measures such as student engagement and parent satisfaction). Using these frameworks should provide considerable insight into the performance of Denver’s charter schools. And what they show us is a significant different in school quality.

On the growth model, adjusted based on the number of tested students in each school, Denver’s charter schools outperformed their traditional peers in academic growth by 15 percent, with an aggregate median growth percentile of 61.3 versus 53.4 (median growth across Colorado is 50). Charter schools scored higher at every school level, and in all subjects, with a single exception. The gains were stronger in the secondary grades (particularly in math and writing) with differences of up to 30 percent.

Denver’s School Performance Framework (SPF), similarly adjusted for school size, also showed a similar 15 percent gap in academic growth among charter and traditional schools – and both schools served equal percentages of students in poverty. However, charter schools excelled even further on the SPF’s non-academic metrics, with astounding differences in measures of student engagement (43 percent higher), parent satisfaction (27 percent), and re-enrollment (16 percent) (see data section below). These gains extended across all grade levels.

The final statistic — a school’s re-enrollment percentage — is particularly interesting.  Among the many unproven claims against charter schools is that they filter out low-performing students.  It turns out that charters have a far better track record of retaining kids.

Here is a full summary of the charter and district data from both frameworks.

Even their strongest proponents agree that charter schools are not a panacea for all of public education, and there are many external and societal factors impacting schools that also badly need our attention. More research should continue to look at the similarities and differences — but the direction here is clear. Across a growing body of evidence and several years of data, the performance of Denver’s charter schools surpasses their traditional peers. Superman is not coming to Denver, but the charter schools here are doing very, very well.

Extra Credit: The Data

If one wants to dig into the details, here is a larger discussion of the data:

CREDO Study: The 16-state CREDO study, which looked at five years of data ending in 2007-08, and to which charter detractors regularly refer, has received its share of criticismover its methodology.  I don’t know it if it useful to revisit that debate, but what is inexcusable is that the same people who cite this study for evidence against Denver’s expansion of charters completely ignore its local conclusions.  The CREDO study, which used only charter schools in Denver for its statewide comparison, explicitly found and concluded that these schools performed “significantly better” than their peers (see their own press release). To argue the whole of the study while not acknowledging the most relevant part is patently absurd, even for partisan political hacks.

Colorado Growth Model: Using this 2010 data, I did a weighted average based on the number of students in each school who took the CSAP. The growth model splits grade levels neatly into K-5, 6-8, and 9-12 – so if a school is a 6-12 program, the growth model counts it as two different schools (a 6-8 middle and a 9-12 high school).  A similar division happens with K-8 schools. This allows for a more precise comparison by grade.  Under this formula, Denver has 135 district schools and 26 charters (which comprise 12% of students of all students taking the CSAP).

Remarkably, charter schools did better on academic growth in every subject and school level, with the single exception of elementary school math. Aggregated across all 161 schools, charters received higher median growth percentile (MGP) scores in reading (+4.8), in writing (+9.4) and in math (+9.3) for an average difference of +7.8 points, or almost 15% higher:

Breaking it down by grade levels, in the elementary school grades (charter students composed 9.3% of tested students), charters did slightly better on average (+1.4); better in reading (+1.1) and writing (5.7) and worse on math (-2.6).

In middle school grades, (14.8% charter students) the differences in median growth percentiles were stark: reading (+7.9), writing (+12.1), and math (+15.5), or a double-digit average of 11.8 points better.  This is a percentage improvement of between 15% and 30%. The district’s lowest scores were in the middle school grades, suggesting that the renaissance in Denver’s middle school years is primarily driven by charter schools.

High school scores (8.7% charter students) were also positive: reading (+1.0), writing (+7.2) and math (+10.1), or an average of +6.1 points or 11% improvement.

School Performance Framework: Note first that this is based on 2010 SPF data (which covers the 2009-2010 school year), which is different than the recent 2010 count day datalisted at EdNews.  The 2010 SPF lists 18 charter schools and 114 district schools. Using this data, I again adjusted scores based on enrollment (so that each school provides a weighted average in its category). I did not include alternative schools in either group.

Across the entire city, district schools enrolled 67,203 students in 2009-2010, while charters had 6,105 (or 8% of the total). The percentage of students in poverty is very close: 73% to 72% FRL. However, in the aggregate on the SPF, charter schools did considerably better on growth (+8 points); status (+13), reenrollment (+13), student engagement (+17), and parent satisfaction (+12).

But to look even closer, Denver has 68 traditional (K-5) elementary schools and just one charter elementary school, which makes any comparison for K-5 meaningless. What happens if we subtract all 69 of these elementary schools and look again at the aggregate SPF metrics? You get this:

Without elementary schools, the relative performance of charters improves even further.  The percentage of students enrolled in charter schools rises to 14%, and the percentage of FRL students is the same (71%). However charter schools receive higher marks across the board — on quantitative academic criteria (+13 growth, +16 status), on re-enrollment (+15), as well as on the qualitative aspects of student engagement and parent satisfaction (+21 each).  These are remarkable and meaningful differences, and as close to a viable district-wide comparison as I think we can get.

What happens if we continue to drill down into specific school grades, comparing K-8 schools, 6-8 middle schools, 6-12 schools, and 9-12 high schools? Well, of the eight academic criteria, charter schools outperform district schools in seven. Charters also do better in every measured level in both student engagement and parent satisfaction. I won’t cover all levels (you can see the full results in the link above); however let’s look at one particular segment: middle schools.

There are 12 district middle schools, and 5 charters (I included KIPP SP, which is listed as K-8 but only offers grades 5-8). The 14% of middle school students in charters provide a reasonable volume for comparison.  What are the results?

The academic differences are remarkable: +23 points on growth, and +18 on status — and charters have 17% more students in poverty.  Charter middle schools also do far better on student engagement. Two of the five charter schools did not have re-enrollment data, and two also did not have parent satisfaction, so I did not do a comparison for either.  But the schools who did report re-enrollment and parent satisfaction were higher than the district school mean. The academic data here is so strong, it makes me question how much of DPS’s recent middle school success is due to the impact of charters.  I suspect it is considerable.

Charter 6-12 schools also had double-digit scores: growth (+25%), status (+19), student engagement (+19), and parent satisfaction (+39). Reenrollment was lower (-13), but this was based primarily on just one school with a particularly low score (and which is being reconfigured).  FRL was comparable with charters at 58% and district schools at 60%.

K-8 schools also showed higher scores across the board for charters: growth (+8), status (+13), reenrollment (+24), student engagement (+13), and parent satisfaction (+17), however they did so with 11% less FRL students.

Where did charter schools fail to outperform district schools? Only in 6-12 high schools — which had the fewest number of charter students at any level and composed just 4% of the total — and in one category.  Charter schools lagged in growth (-10), but were higher in status (+5), and student engagement (+33), and had an FRL population 15 percentage points higher.

That’s the data, which included results from 2002 to 2010.  Of course, the arguments about charter school performance in Denver is, at many levels, based on political calculations and interest groups who have priorities other than the educational outcomes for students.  Those people and groups will continue their protests regardless (in fact, it would not surprise me if they called for a repeal of the metrics themselves). Which, or course, does not change the data: Denver’s charter’s are doing very, very well.