Over the last 15 years, the most popular theory about racial differences in IQ has been “Stereotype Threat.” The New York Times summarized it in its 2009 Year in Ideas featurette on the purported “Obama Effect”—the widespread assumption that the politician’s success might raise black test scores:“In 1995, two Stanford psychologists, Claude Steele and Joshua Aronson, demonstrated that African-American college students did worse on tests of academic ability when they were exposed beforehand to suggestions that they were being judged according to their race. Steele and Aronson hypothesized that this effect, which they labeled stereotype threat, might explain part of the persistent achievement gap between white and black students. In the years since, this idea has spread throughout the social sciences.”
Stereotype Threat is a beautiful theory for explaining the racial IQ gap. Indeed, I myself have often felt there might even be a little bit of truth to the idea that expectations matter—even though common sense suggests that incentives matter more.
But now, it turns out that the vaunted evidence for this wildly popular concept rests heavily upon another Effect, the File Drawer Effect—defined as “the practice of researchers filing away studies with negative outcomes”. We seem to have another Climate Research Unit scandal on our hands.
A researcher, who doesn’t want his name or any potentially identifying information mentioned, for unfortunately obvious career reasons, recently attended a presentation at a scientific conference. Here is his summary of what he heard:“One talk presented a meta-analysis of stereotype threat. The presenter was able to find a ton of unpublished studies. The overall conclusion is that stereotype threat does not exist. The unpublished and published studies were compared on many indices of quality, including sample size, and the only variable predicting publication was whether a significant effect of stereotype threat was found. …
“This is quite embarrassing for psychology as a science.”
Here’s the abstract of the presentation he heard (see p. 68 of the PDF)“Numerous laboratory experiments have been conducted to show that African Americans’ cognitive test performance suffers under stereotype threat, i.e., the fear of confirming negative stereotypes concerning one’s group. A meta-analysis of 55 published and unpublished studies of this effect shows clear signs of publication bias.”
[Stereotype threat and the cognitive test performance of African Americans, by Jelte M. Wicherts & Cor de Haan, University of Amsterdam]
In other words, if a study doesn’t find the existence of stereotype threat, it’s less likely to see the light of day. Positive results are more appealing to journal editors, and politically correct positive results are loveliest of all. In contrast, how much of a market is there for punching holes in society’s fondest hopes?
The Dutch researchers continue:“The effect varies widely across studies, and is generally small. Although elite university undergraduates may underperform on cognitive tests due to stereotype threat, this effect does not generalize to non-adapted standardized tests, high-stakes settings, and less academically gifted test-takers.”
Note that “Stereotype Threat” mostly seems to exist in settings where test-takers, such as “elite university undergraduates,” are smart enough to pick up on researchers’ hints about what results they hope to publish. In the marketing research industry, it’s well known that survey respondents tend to respond with the answers that they surmise the pollster wants. Human beings like to be cooperative if it doesn’t cost them anything. Similarly, it’s likely not hard for black students at top research universities to gather that they can benefit their professor with a publishable paper just by working less diligently on the meaningless test he's given them.
Moreover, “Stereotype Threat” doesn’t seem to exist where the test is important enough to matter to the students.
January 16, 2010
From my VDARE.com column: