In “What PISA Can’t Teach Us,” Carnoy, Garcia and Khavenson directly deliver three key findings, and implicitly deliver a fourth.
- “There is no causal evidence that students in some Asian countries, for example, score higher on international tests mainly because of better schooling.” As I’ve pointed out previously, similar school systems around the world are producing similar, disappointing scores. Echoing a recurrent theme, the #1 South Koreans are not happy with the performance of their school system. Relentless performance pressure, not better formal schooling, seems to be a much better explanation for the ~11% difference between U.S. and South Korean PISA scores.
- In the U.S., because K-12 governance and funding policy is made at the state level, it is inappropriate to compare a U.S. average test score to the world’s other school systems. The U.S. actually has 51 school systems. The same thing is true for some other countries; for example, Canada, where provinces make primary and secondary education governance and funding policy.
- There are significant differences between the 51 U.S. systems. Not only do the scores differ, but score trends differ. For example, two states were similar a few years ago, but one shot ahead. For central plan improvement, it would helpful to compare those two states’ recent policy changes. And the differences between the scores can be used to explore which policy differences between the states account for the score differences. But, state comparison can ONLY reveal the importance of ways in which the fifty states and DC actually differ.
- So, implicit in the Carnoy et all findings is that U.S. state comparisons can only inform central plan improvement. All fifty states and DC rely on the political process to decide which instructional approaches will be produced, how, where, and for whom. As the international comparisons and within-country state comparisons reveal, central plan improvement has a low upside. The best in the U.S., and the best in the world, are not that far above the “Nation at Risk” — level average U.S. performance. Internationally, we can find some price system presence to facilitate the with/without prices comparisons — for example, the Sweden-Chile comparison I suggested — to provide some empirically-grounded insight into the nature of differences between centrally planned and price system-orchestrated schooling. We’ll also need the will and wisdom to rely, when necessary, on theory-based evidence, and experience from outside modern formal schooling, to move us forward towards productive, transformational school system change.