What does the SAT test?
The SAT tests a) General intelligence b) Academic aptitude c) Test-taking skills d) Nobody really knows
NEARLY EIGHT DECADES after Carl Campbell Brigham, a Princeton University psychologist, administered the first SAT to a small group of pencil-chewing applicants to Ivy League schools, a debate still rages about what the ubiquitous college-admissions exam actually measures. Do those infamous scores, ranging from 200 to 800 on separate verbal and math portions, capture IQ, reasoning ability, academic "aptitude" (whatever that is), socioeconomic status, test-taking facility -- or some other quality?
The SAT's makers themselves betray a certain uneasiness on this question, as the evolving name of the test suggests. The SAT began life as the Scholastic Aptitude Test in 1926, when a mere 8,000 students took it. In 1994, the Educational Testing Service (ETS), the company that produces the exam, changed the name to Scholastic Assessment Test and began, confusingly, to refer to the old "achievement" tests in subjects like US history or biology as "SATs" as well. These days, the company advises calling it simply "the SAT."
Now, however, just before the SAT undergoes another of its periodic transformations -- a new version of the exam, referred to as the "New SAT," will be unveiled next spring and will include, for the first time, a writing portion -- a study published in the June issue of the journal Psychological Science claims to prove that the current SAT is, in the end, an IQ test. The authors, psychology professor Douglas K. Detterman of Case Western Reserve University and graduate student Meredith C. Frey, argue that the SAT actually measures "general intelligence" -- or g, in the argot of psychometricians -- more effectively than some IQ tests wielded by psychologists (which often ask test-takers to do things like discern patterns in strings of numbers). The authors even provide formulas for converting SAT scores into IQ scores. (See sidebar, opposite page.)
Frey and Detterman came to these conclusions by analyzing two data sets. Using existing records, they examined the scores of 11,878 students who had taken the Armed Services Vocational Battery, designed in part as a probe of IQ, in 1979. Some 917 of these students had also taken the SAT -- and the results of the two tests correlated very closely. They also gave the Raven Test of Progressive Matrices (a very abstract IQ test, testing pattern recognition) to 104 Case Western students who had valid SAT scores on record at the university. Again, these students' IQ scores and SAT scores were tightly correlated.
Frey and Detterman offer several innocuous reasons that their finding might be useful. If someone who has never taken an IQ test suffers brain damage or mental deterioration, for example, his or her SAT score can be used as a rough indication of pre-injury cognitive skills. Psychologists might also want to know the IQ scores of students they are studying in their experiments, and now they can do so without administering a whole new battery of tests.
Yet linking SAT scores to IQ also opens a Pandora's Box of issues that ETS prefers to keep closed. For one thing, researchers believe that IQ -- g is the more precise term -- rises through one's teenage years but stays largely fixed after that. The new study implies that people with low SAT scores didn't just flub a test or go to a lousy school: They are burdened with low intelligence.
IQ tests are also controversial because they have been used in the past to "prove" gaps in intelligence between racial groups and to explain, and even justify, class stratification. In their 1994 book "The Bell Curve," for example, Charles Murray and Richard J. Herrnstein argued explosively that most social-engineering efforts, including affirmative action and wealth redistribution, were doomed because of unbridgeable IQ gaps.
"ETS, politically, has always tried to distance itself from the IQ debate," says Nicholas Lemann, author of "The Big Test: The Secret History of the American Meritocracy" (1999) and now dean of Columbia's Graduate School of Journalism. "Of all the subjects that make ETS uncomfortable, this would be No. 1 on the list."
. . .
In "The Big Test," Lemann traced the SAT's evolution directly out of the IQ tests administered to American soldiers in World War I. The test, he argued, had "changed remarkably little" since then, despite public-relations spin to the contrary.
ETS's defenders -- including Time Magazine's John Cloud, who penned a cover story on the controversy last fall -- say that the SAT departed from the IQ-test tradition as early as the 1930s, when it shed its most abstract and obscure categories of questions. Now the test's makers say it measures "reasoning" -- the sorts of skills one needs to do college work.
And it does it well, according to the College Board, which still insists, together with ETS, that the SAT is the best single predictor of performance in college -- better even than high-school grades, although combining the two provides the best results of all. The SAT, moreover, can theoretically identify bright kids in weak schools.
Critics, though, launch salvos against the SAT's predictive value each year. In 2001, Richard Atkinson, then the president of the University of California system, proposed shifting to college admissions based on achievement-test results, which would encourage students to study in class, rather than waste time in expensive test-prep classes. Yet amazingly the College Board, along with those psychologists who think the SAT measures IQ, maintains that the SAT is basically uncoachable. The best research, they say, shows coaching leading to improvements of only 30 points per section. The test-prep company Princeton Review, meanwhile, guarantees a 100-point improvement in total score -- and doesn't seem to be going out of business doing so.
The new study is not likely to change anyone's mind about the SAT. In fact, both the people who hate it the most, and those who love it, claim the study confirms their view. Those who think general intelligence is a highly important quality that can be assigned a single number -- call them the "IQ hawks" -- say the study confirms something obvious: Any test of reasoning also tests IQ. And, they say, IQ predicts success in college just as SAT scores do.
"As many people in the field have pointed out, it's the very success of mental tests that makes [critics] hate them," says Linda Gottfredson, a professor of education at the University of Delaware. "They do a dirty job. They distinguish people according to relative differences in ability. While that's what a meritocracy is supposed to do, we somehow imagine that we can have our meritocracy and equality, too."
Yet those on the other side of the IQ divide say the study confirms just how narrow a set of skills the SAT and other IQ tests assess. "It's a truth-in-advertising deal: People ought to know what they are getting," says Robert J. Sternberg, a psychology professor at Yale who has been working with ETS on creating standardized tests that measure creativity, practical skills, and ethics.
For its part, the College Board plays down the relevance of an IQ-SAT correlation. "It's not really a relevant question to what we do," says Wayne Camara, the College Board's vice president for research and psychometrics. "The job of the College Board is to prepare kids for college, and we use the SAT to predict how well kids do in college." IQ tests, the SAT, state academic tests, and achievement tests all "correlate quite highly," he points out. "But that doesn't mean that they measure exactly the same thing. Height and weight also correlate quite highly." But Camara doesn't deny that both the SAT and IQ tests measure some aspect of general intelligence.
ETS has pushed the New SAT a few steps down the road toward becoming an achievement test -- adding grammar questions, for example. But the IQ hawks and IQ detractors are still likely to view it as an IQ test in disguise.
Nicholas Lemann says the whole SAT-IQ debate is mainly a semantic one. "I think the country would have been less likely to accept [the SAT] as a mass instrument if it were presented as an IQ test. If it is presented as measuring something more complicated and mysterious, like `developed ability,' it's an easier sell."
Christopher Shea writes the Critical Faculties column for Ideas. E-mail: firstname.lastname@example.org.