Personality tests are everywhere -- from the workplace to the courtroom. But critics say the tests themselves don't pass the test.
DO YOU PREFER a bath to a shower? Are you fascinated by fire? At parties, do you sometimes get bored, or always have fun? Do you sometimes feel like smashing things? Do you think Lincoln was greater than Washington? Do you feel uneasy indoors? Do you think questions like these tell us anything meaningful about ourselves, or do you think they're nothing more than parlor game fodder?
Regardless of how you answer that last one, the fact is that personality tests featuring questions like those are everywhere these days. The Minnesota Multiphasic Personality Inventory, or MMPI, is taken by as many as 15 million people a year and used to screen applicants for jobs from police officer to nuclear technician to priest. Eighty-nine companies in the Fortune 100 use the Myers-Briggs Type Indicator to determine how and with whom their employees work best. The Rorschach test, the granddaddy of them all, is used diagnostically by eight out of 10 psychologists and routinely submitted as evidence in child custody cases, criminal sentencing, and emotional damage lawsuits. Online dating sites even use personality tests to help match prospective couples.
Clearly, there's something about the elusive notion of personality, and the possibility of capturing it, that draws us to these tests. But an increasingly vocal group of critics is fighting this testing tsunami, arguing that many of the tests themselves have not been tested and that their unscientific conclusions may do far more harm than good. Last year, in "What's Wrong with the Rorschach? Science Confronts the Controversial Inkblot Test" (California), four psychologists dismissed the Rorschach test as having no more validity than "tea-leaf reading and tarot cards." In "The Cult of Personality: How Personality Tests Are Leading Us to Miseducate Our Children, Mismanage Our Companies, and Misunderstand Ourselves" (Free Press), due out later this month, Annie Murphy Paul, a former senior editor of Psychology Today, charges that personality tests "are often invalid, unreliable, and unfair" and that their prevalence has grave consequences, not least the distortion of the very idea of human nature to fit their arbitrary dictates.
Each of the major personality tests has its impassioned defenders. But the dispute hasn't reached the clamorous level of, say, the argument over intelligence testing or the SAT. It may only be a matter of time, though. After all, the debate places two of our most powerful, and perhaps irreconcilable, impulses -- our desire to define ourselves in simple, concrete terms and our need to carve out a personal identity uniquely our own -- in stark opposition.
. . .
No personality test is more iconic than the one created by the Swiss psychiatrist Herman Rorschach [see sidebar]. In 1917 he hit upon the idea of showing subjects a series of 10 abstract, bisymmetrical inkblots (the same ones used today) and asking them what they saw. The blots' very ambiguity, he believed, created a sort of perceptual vacuum into which his subjects' inner tendencies flowed. Not only what they saw (morbid or unusual imagery, for example) but how they saw -- whether they picked out movement or tended to privilege color over form -- revealed their modes of understanding and, in turn, who they were. Rorschach believed the test could not only diagnose mental illness, but provide a detailed map of all "the numberless nuances of personality."
The test achieved wide currency in the United States, but came under attack in the 1960s for its lack of statistical support. Today, the Rorschach might be only a historical footnote if not for a psychologist named John Exner, then a professor at Bowling Green State University, who in 1974 introduced a complex scoring matrix for the test called the Comprehensive System. It involved more than 140 components, including a Depression Index, an Egocentricity Index, and an Aspirational Ratio. Today, all Rorschachers swear by it.
David Medoff, a psychologist and codirector of Massachusetts General Hospital's Children and the Law Program, is a staunch supporter of the test. He has used it to provide expert testimony in child custody disputes, sex offender evaluations, civil suits, juvenile delinquency cases, and other proceedings. "The Rorschach is routinely relied upon in the forensic realm," he said in a recent interview. "It's alive and well, and it's adding to the information base that judges and juries are using to make decisions."
While Medoff is quick to add that the Rorschach is only used by psychologists and the courts as "one test among many," that disclaimer does little to mollify the test's critics. In legal and clinical settings alike, they argue, Rorschach evidence still carries plenty of weight, and as a result, decisions that can ruin lives or cost millions are being swayed by pernicious pseudoscience. A recent survey of forensic psychologists in the journal Professional Psychologist rated the Rorschach "unacceptable" for most courtroom uses.
Howard Garb, one of the coauthors of "What's Wrong with the Rorschach?" and the head of psychological testing for the United States Air Force, believes that "only 10 percent" of Exner's Comprehensive System meets even the most basic scientific standards. Examination of the data in over 30 Rorschach studies, he argues, shows that the tests have a marked tendency to label healthy people mentally ill. In one 2000 study, for example, 100 mentally sound schoolchildren were given the Rorschach and the majority received scores indicating that they were borderline psychotic.
As Scott Lilienfeld, an associate professor of psychology at Emory University and another "What's Wrong with the Rorschach?" coauthor, puts it, "People have looked and looked and looked at whether these scores relate to the kinds of psychological traits and behaviors that they're supposed to, and most of the studies show no effects or no relationships. And even when some positive findings pop up here and there they often are not repeated." Those few positive findings, he says, are usually from researchers affiliated with Rorschach Workshops, an organization Exner heads in Asheville, N.C.
Exner acknowledges that some parts of the Comprehensive System aren't as solidly supported as others and cautioned that the test should be used as much as a "descriptive" as a "diagnostic" tool. He also concedes that his Depression Index doesn't really measure depression -- what it does measure, he says, is "emotional disarray" -- and admits that he has considered changing its name. But the other charges against the test are "nonsense," he says, and points to a sheaf of peer-reviewed studies -- some from Rorschach Workshops, some not -- backing up the claims of the Comprehensive System.
Mass. General's Medoff, for his part, sees Rorschach opponents as scandal-mongers. "They've created this illusion of a controversy," he says. "They've singled out the Rorschach to the degree that few other psychological tests have been scrutinized. No psychological test is perfect."
What's striking, though, is that not even Exner pretends to have any understanding of why the test works -- why, for example, a tendency to see in terms of color rather than form or to see pairs of images (which would appear normal in a symmetrical inkblot) might, as he claims, consistently predict depression or narcissism or even psychosis. Indeed, despite the purported statistical grounding of the test, it retains a sense of mystery -- and that's what seems to so infuriate its critics.
. . .
Largely because of its complexity, the Rorschach hasn't spread beyond psychology (and even there it's under assault from HMOs and insurance companies reluctant to fund a time-consuming test whose validity has come into question). Today, the testing boom is being fueled by user-friendly pencil-and-paper tests better suited to the business world.
Like the Rorschach, the Minnesota Multiphasic Personality Inventory, or MMPI, was designed to spot the mentally ill. Introduced to the world in 1942 by Starke Hathaway, a psychologist, and J. Charnley McKinley, a neuropsychiatrist, both of the University of Minnesota, the MMPI contained 504 statements -- such as "Several times a week I feel as if something dreadful is going to happen," "I wake up fresh and rested most mornings," "There is something wrong with my sex organs," and "Often I feel as if there were a tight band around my head" -- to which the subject was instructed to respond "True," "False," or "Cannot say." At the time the test was unique in proposing to determine pathology not by looking at the substance of the responses, but simply by comparing them to those of a control group of 724 presumably sane Minnesotans (the relatives and friends of the patients at the university's hospital) who had also taken the test.
The MMPI is by far the most popular psychological test currently used, and almost all psychologists, and even most test skeptics, place some stock in it. Historically the big battles over the MMPI have been waged less over its accuracy than its invasiveness. Within a decade after its creation, the MMPI had escaped the mental ward and, as Annie Murphy Paul writes in "The Cult of Personality," by the 1960s it was being given as often to people in job settings and court cases as to psychiatric patients.
As a result, the MMPI and its imitators came under assault from sociologists like William H. Whyte, who saw the tests as helping to create and perpetuate the oppressive group-think of the mid-century "organization man." In 1966, Senator Sam J. Ervin Jr. of North Carolina introduced a bill to sharply curtail the government's use of the MMPI and similar tests, decrying it as a wanton invasion of privacy comparable to McCarthyism. Ervin's bill didn't pass, though, and the MMPI only grew in popularity. In 1989 the test was "re-normed" with a new control group and some of the more bizarre questions were excised.
But the breadth of the MMPI's use still stirs controversy. The past decade has seen a series of lawsuits over the role of the test and its knock-offs in the job interviewing process (or, in one case, as a condition of welfare benefits), where its inventors never intended for it to be used. According to Brad Seligman, a lawyer who brought three such cases under the California state constitution's protection of privacy and fairness in employment, some of the questions -- about one's sex life, political views, religious beliefs, and even bowel movements -- are "highly intrusive." Furthermore, he notes, there is no research linking the test results to job performance, especially in the strict pass/fail way it was used by employers. In all three cases, the courts agreed.
. . .
While the MMPI is more widely given, and the Rorschach better known, perhaps the most beloved personality test is the Myers-Briggs Type Indicator. Based on Jungian psychoanalytic theory, it was developed in the 1940s and `50s by a bored, bright Philadelphia housewife named Isabel Myers, with some input from her mother, Katharine Briggs. Unlike the Rorschach and the MMPI, it was expressly designed not to diagnose psychopathology but to describe normal personalities. Today, the four Myers-Briggs axes -- Extraversion-Introversion, Sensing-Intuition, Thinking-Feeling, and Judgment-Perception -- are a reflexive lingo to a whole generation of corporate managers, consultants, and headhunters.
The test's popularity stems from the basic belief that "personality influences how you interact with other people, so having a detailed understanding of personality allows you to adjust for individual differences," says David Thomas, a Harvard Business School professor who has studied its use in corporate America. The questions the test asks are straightforward: "Do you usually: A) share your feelings freely, or B) keep your feelings to yourself?" or "Do you: A) rather prefer to do things at the last minute, or B) find that hard on the nerves?" None of its 16 personality types are considered more healthy or normal than the others. Because it's free of the language of mental illness, the Myers-Briggs manages to classify without stigmatizing.
But for all its ubiquity in the boardroom and as an online quiz, the test is generally ignored or ridiculed by psychologists. Robert Hogan, a former psychology professor at the University of Tulsa who now runs his own testing company, Hogan Assessments, says, "I used to use [Myers-Briggs] as an icebreaker. People like taking it, and when you get the results back you feel good. But it has the intellectual content of a fortune cookie."
"There's no evidence," Hogan concludes, "that it predicts job performance or any meaningful non-test outcomes."
Defenders of the Myers-Briggs say these criticisms are unfair. Steven Reiss, a professor of psychiatry and psychology at Ohio State University (and inventor of a similar test, the Reiss Profile) admits that the test is marred by scoring problems and outdated theoretical baggage. But, he says, it's still much more accurate than the more respectable MMPI. "Academic people have not looked at it because it's based on Jung," he says. "So it's based on bad science, but it actually works."
Ultimately, the biggest value -- and danger -- of the Myers-Briggs may be the way it lubricates social interactions. Harvard's David Thomas concedes that in its current corporate usage, the value of the test can stem as much from getting everyone in a room "speaking a common language" as from any real insights about personality. In other words, it's an exercise in conflict resolution and group therapy, a way to get substantive disputes to dissolve in a warm bath of psychologically tinged language about different types, perspectives, and styles.
For Annie Murphy Paul, this isn't necessarily a good thing. In an interview, she pointed out that Myers-Briggs is distinctive in that it's not only driven by institutional needs, but sought out by individuals. "People really, really like it -- they latch on to the results," she says. For her, the problem with the test, and with all personality tests, is that it "limits and stereotypes the ways institutions think about individuals and the way individuals think about themselves." Personality types, she says, are "one-dimensional labels."
But the tests -- and the controversy over them -- aren't likely to go away anytime soon. For most of us, there is something tantalizing about the prospect of easily digestible self-knowledge -- just as there's something galling about a test that claims to tell us exactly who we are.
Drake Bennett is the staff writer for Ideas.