Dicing the Data on Charter Schools

John Mooney | November 12, 2010 | Education
What do the numbers say about charter school performance? It all depends on who you ask

The debate over the performance of charter schools isn’t new in New Jersey, but it’s growing louder — and possibly more strident — with Gov. Chris Christie’s vow to expand them in number and scope.

But a related dispute also is being carried out both in and outside of the state over how to evaluate the performance of charters, an argument that pits academia against advocacy.

Running the Numbers

Over the past several months, a flurry of data has been parsed and prodded to determine how New Jersey’s 73 charters schools fare against the state’s traditional public schools. And often depending on the numbers being used, the answers have varied significantly.

The state’s charter school advocates, both inside the state Department of Education (DOE) and within the schools and their umbrella organizations, have pointed to a handful of charters as some of the highest-performing schools in their respective cities, if not in the state.

Compared with district averages, most charters have done better, they say, especially in eighth grade tests, one of the state’s prime benchmarks.

Critics and at least one researcher out of Rutgers have dug deeper into the same data and pointed out that any such analysis is shallow, at best.

They say the supporters’ numbers fail to account for the vastly different populations between charters and their traditional counterparts, with the latter serving students who are poorer and with more special needs, both in terms of disabilities and language.

The Middle Way

And then this week, the Advocates for Children of New Jersey (ACNJ), a long-time advocacy group, came out with its annual report on Newark children, this time including a charter school section that landed somewhere in the middle for the dozen charter schools in that city.

Some charters far exceeded the average achievement levels; others fell way short.

One of those involved in the research, part of the Kids Count report, called it an “honest look at the data.”

“It is no means definitive or the end; we recognize there is other research out there,” said Nancy Parello, ACNJ’s communications director. “Are there other ways to dice this up, absolutely. But looking at it in the averages, that is of value, too.”

But it has often been how the numbers are diced that has sparked the harshest dispute.

School Finance 101

Bruce Baker, an associate professor in education at Rutgers, has provided most of the ammunition for critics in terms of putting forward precise, sometimes esoteric, data critical of charter schools’ achievement claims.

Through his blog, School Finance 101, Baker has pointed out the clear disparities in the populations of both charter schools and traditional schools, often in the same neighborhoods.

Claiming that charters “cream” the best students from traditional schools, he cites lower special education rates and lower rates of students receiving free lunch subsidies, one common measure of poverty in schools.

Baker also stresses the clear distinction between free lunch subsidies and reduced lunch subsidies, which both charter proponents and the ACNJ used.

“Seems like a subtle difference for the lay reader and one that might not sink in right away,” Baker said in his latest blog post. “But, it can actually be an important distinction in this type of comparison.”

He adds: “Most Newark Charter Schools, especially the frequently touted high performers, have very low relative rates of children below the 130 percent poverty threshold [for free lunch subsidies].”

Still, none of this has settled the dispute, as the state continues to press new measures to help charter schools expand. A bill that would extend authorizing powers for new charter schools appears close to approval in the legislature.

Sidestepping the Issue?

The state Department of Education, for its part, has avoided weighing in entirely. Last week, acting state education commissioner Rochelle Hendricks said her department needed more time to complete its own analysis, leaving some State Board of Education members incredulous.

One suggested that if the state were truly serious about comparing apples and apples, it would compare students who attended charter schools with those who applied but failed to get in through the school’s lotteries.

“It is so important to have data that is truly comparable,” said board member Dorothy Strickland, herself a Rutgers education professor.

Baker said yesterday in an email that he hopes the state takes up Strickland’s advice, saying such a comparison is attainable.

“One reason why I do all of this is to reveal what type of data we would need to address the questions in a more precise manner,” he wrote.