Few news beats offer a monsoon of data equal to that in education.
Numbers frame every argument and initiative. But Mark Twain said it best: “There are three kinds of lies: lies, damned lies and statistics.”
In other words: Beware the study with provocative findings advertised by a screaming headline that supports one silver-bullet approach or another. A piece from The Atlantic makes this point eloquently, quoting Maine math teacher Tracy Zager:
Public education has always been politicized, but we’ve recently jumped the shark. Catchy articles about education circulate widely, for understandable reason, but I wish education reporters would resist the impulse to over-generalize or sensationalize research findings.
We hear that.
In May, a post on the Education Lab blog threw cold water on the latest headline-grabbing study to suggest that parent involvement has little bearing on student achievement. As reporter Linda Shaw noted: the research relied on federal surveys — capturing only official, PTA-style involvement and missing more innovative, in-classroom practices that have been shown to be effective.
As Times reporter John Higgins wrote last December, “Ice cream does not cause drowning.” He was making a point about hidden variables: Even though drownings increase at the same time that ice cream sales go up, “the lurking variable is a hot summer day, which boosts ice cream sales and swimming.”
This stuff seems elementary. But The Atlantic piece is an important reminder of lessons many of us learned in basic science and sometimes forget in the frenzy to stay abreast of every seemingly important development: Avoid extrapolating too much from a single study; view small-sample findings with a skeptical eye; correlation does not imply causation.
For a more detailed list on what to look for (or avoid) when perusing research, scroll through 20 tips published by the weekly science journal, Nature.