Last Friday, David Brooks had an op-ed in the New York Times with the headline The Harlem Miracle, discussing an evaluation of the schools run by the Harlem Children's Zone. While lots of people have been excited by the concept of the HCZ -- it's the basis for the "Promise Neighborhoods" idea that Obama talked about in the campaign and included in his budget -- there hasn't been any hard data about effectiveness until now. Here's the underlying study* which really is quite exciting.
The main findings of the study are:
- For the middle school students, there were really enormous gains in math scores, although they took several years to kick in. The gains in language arts scores were much more modest. These findings are based on comparisons between those randomly selected for admission and those who applied but were randomly denied, so they're about as strong as you get.
- The elementary school impacts were stronger on language arts, somewhat smaller on math, but still impressive. Because few students who applied for the elementary schools were denied admission, these findings are based on a different statistical approach (instrumental variables), which is somewhat less reliable.
- The authors did not find any significant effects on test scores for graduates of either Baby College or Harlem GEMS (the preschool program run by the HCZ). They also note that the middle school impacts were as strong for kids who lived outside of the Zone as for those who were in it, suggesting that the full community package was not essential to the model.
So, what does this mean? To start with, it refutes the claims of some that there's nothing you can do to help these kids do better in school and society. (The strong version of this claim is that IQ is genetic and can't be affected by anything you do, the weaker version is the claim that by the time the kids are in middle school it's too late.)
Brooks uses this finding to argue for "an emerging model for low-income students" where "schools create a disciplined, orderly and demanding counterculture to inculcate middle-class values." The thing to notice here is that Brooks is lumping HCZ and KIPP together. Both models certainly share some features, including extended school days and years, and very high expectations.
However, if you read Whatever it Takes, one of the main themes is that Geoffrey Canada (who runs HCZ) was constantly fighting his board, who thought they should just bring KIPP in to run these schools. Canada felt that KIPP was too focused on rescuing a few students -- and encouragin these students to define themselves in opposition to the neighborhood culture -- whereas he wanted to change the neighborhood culture. He also fought against explicitly teaching behaviors like making eye contact, arguing that no middle class school does that. So, I don't know whether Canada gave in on these points, or if Brooks is distorting HCZ to fit his agenda.
But presumably, other people do have a good idea of what exactly is going on the HCZ schools. Is this model then broadly replicable? That depends on a bunch of questions:
- Are there enough good teachers out there who are willling to work in low-income neighborhoods, with the kind of hours required, and under intense pressure to achieve good test scores? (HCZ had extremely high turnover of teachers.) And are we, as a society, willing to pay enough to recruit teachers to do this?
- Are the kids willing to work as hard they have to to succeed in this model? To give up afternoons and summers and weekends, and to work harder in school than they ever have before?
- How much of this success is dependent on Canada himself? His personal charisma is clearly part of what made both teachers and students willing to work so hard. And his personal story makes him a very convincing messenger for the idea that if you work hard you can succeed, even coming from poverty in Harlem. No one is going to give up their weekends and summers unless they're convinced that it will make a difference.
* It drives me crazy that the Times never includes links to underlying sources. But it cracked me up that Judith Warner's blog last week included a linked definition for "muffin top."
Others on this column: