We have to carefully use “evidence” to assess policy options, which sometimes means rejecting alleged evidence. And in this “Mad, Mad World” of school system reform controversies we often have it backwards. Facts that constitute relevant evidence are widely ignored for policies that seem like they should produce noteworthy results even though they haven’t. For example, we have tried to improve the performance of the current K-12 system with increased per pupil spending, higher standards, smaller classes, recruitment of higher ability teachers and increased teacher training. And even though those policies haven’t produced noteworthy gains, because they seem like they should produce gains, there are repeated failed efforts to apply them in the circumstance in which they have already repeatedly failed to produce the improvements we need. Higher standards and the promise/threat of “increased accountability” seem like they should yield academic gains, but in the current system they have not yielded noteworthy gains. In a different system, the common sense that they should matter might be matched by findings that they do. Because the “roots of the low performance problem” persist, my study of studies demonstrated that the symbolically sensible policy changes, like those listed above, have not mattered much, or at all.
Many facts generated by the current system are not relevant to decisions to apply policies in circumstances radically different from those present right now. For example, many opponents of school choice expansion cite (Ravitch, 2010; Chapter 7) the lackluster, though positive outcomes, of restriction-laden school choice programs, as evidence that they believe should discredit school choice policy options, generally. But the kinds of policies that could provide a large expansion of the menu of schooling options, and leverage entrepreneurial initiative to do it, have not been tried.
Studies of alleged merit pay (not all such programs even attempt to reward measured individual merit) for traditional public school teachers indicate that “merit pay” policies have not produced noteworthy gains. Those results have been cited to challenge reform advocates’ complaints that the current system does not punish failure or reward extraordinarily effective educator performance (“we tried that; it didn’t matter”). But with merit measured by market outcomes, differential teacher pay could be one of the reasons for better academic outcomes when parent/customers assess and reward merit, or punish its absence by enrolling their children elsewhere.
Because the private schools of the current system are generally not measurably much superior to traditional public schools, some analysts believe they have evidence that private school choice expansion of any kind, even market accountability-based transformation of school systems, will not yield noteworthy improvements. But the private schools of a system in which public schools don’t have a public finance monopoly could be, and likely would be, radically different from the private schools that are typical now, and perhaps much more effective than current private schools. However, because transformational change might also improve traditional public schools, the private schools of a new system might still be comparable in measured effectiveness to the public schools of a new system. In addition to possible benefits of competitive pressures, school choice is almost certain to improve public schools by fostering the exit of children for whom the assigned public school was not a good fit. The exit of the bad fits leaves behind a more teachable homogenous set of children (classrooms with few or zero distracting outliers) for whom the public school instructional approach was already at least acceptable, and in some cases quite a good fit.
So, do think about the relevance of alleged evidence. Assert the relevant facts, and challenge unfounded generalizations of facts to circumstances to which they are not relevant.