Dubious journalism on the gender-and-math front

Alex Tabarrok, a George Mason University economist, has got the goods on several leading newspapers and magazines: They inaccurately described the results of a new study of girls' and boys' relative math skills, with at least one compounding its error by taking an unfair potshot at former Harvard president Larry Summers. Summers, Tabarrok notes, could just as easily point to parts of the paper to support his own views.

The study in question was published by Janet Hyde, a psychologist at the University of Wisconsin, and four colleagues in Science late last month. It examined the results of standardized math tests taken by some 7 million students in 10 states, from second grade through 11th.

With the title "Gender Similarities Characterize Math Performance," the article was certainly framed as a contribution to the pro-gender-equality side of the debate. And, indeed, the authors' main conclusion was that, overall, from 2nd through 11th grade, the math-test scores of girls and boys were indistinguishable. That's certainly a newsworthy and important finding, but overall is a key word.

In addition to the main finding, the study also identified precisely the pattern that Summers stressed: Boys show greater variance in math talent. They're overrepresented both in the pool of superstars and among low achievers. This difference in variance was apparent at every grade level.

The Science authors said the variance was "not large," but, as Tabarrok pointed out, it would nonetheless have substantial effects at the far left and far right of the bell curve (i.e., at the extremes). In the paper itself, Hyde and her colleagues use Minnesota 11th graders to explain the phenomenon. In the cohort of white students above the 95th percentile, for example, there are 1.45 boys for every girl. Go above the 99th percentile and the ratio grows to 2-to-1. Go higher, and the discrepancy would only grow further. And Summers's point was that Ivy League physicists are about as high on the ladder as you can get.

However, the L.A. Times, The Chronicle of Higher Education, and Scientific American all stated flat-out that the study found no gender differences even at the very top. A typical line from the L.A. Times: "The study also undermined the assumption -- infamously espoused by former Harvard University President Lawrence H. Summers in 2005 -- that boys are more likely than girls to be math geniuses." (The Globe, for its part, ran a wire story that did not say much about high achievers, though it did suggest the study contradicted Summers.)

Asked about Tabarrok's comments, Hyde makes several points. First, the main finding is still important: For the overwhelming percent of students there is no difference in ability; any girl (or boy) performing at the 95th percentile has the chops to pursue a career in math or tech. Second, even if you accept the Summers-Tabarrok thesis, the lack of women in college physics departments is even lower than you'd predict from the statistical variance. Surely something else is going on.

Finally, and most intriguingly, there's evidence that the higher variance among boys could be a cultural artifact. Among the 11th-grade Asian Americans in Minnesota in her sample, for example, there were actually more women than men above the 99th percentile.

This debate is hardly over -- these are not cut-and-dried issues. But score one for Tabarrok over the MSM. (Says the MSM blogger.)

P.S., 10:23 p.m. Credit where credit's due. Tabarrok says the Wall Street Journal nailed the story. But Andrew Gelman of Columbia says you get the full picture only if you combine the WSJ's take with the New York Times's.

*P.S.S., 8/15/08: I initially neglected to credit the excellent blog xkcd.com for the cartoon.