by Latoya Peterson
This was originally published on 5-17-2011
Look at the Methodology
Whenever you hear the word “study,” start checking for the methodology. Oftentimes, a methodology will reveal more about the study than the summarized results.
A good example of this is a study we were alerted to a year or so ago. The Daily Mail covered a scientific study which proposed that racism may be hard wired into our brains. However, there was an obvious flaw in the study:
All the viewers were white but the researchers believe the results would still have been similar with any other group.
Now, this study wasn’t using basic things, like a sample representative of population. Yet the study authors felt confident in applying the results to everyone.
The same issue pops up in Satoshi Kanazawa’s piece. He actually doesn’t refer to his own research, but another study. And he doesn’t link to the other study, assuming that all readers will know the term “Add Health.” What he refers to is a rigorous, national study…about teen development and health.
The National Longitudinal Study of Adolescent Health (also known as Add Health, the Add Health Study, and the Add Health Survey) is a nationally representative study originally designed to examine how social contexts (such as families, friends, peers, schools, neighborhoods, and communities) influence teens’ health and risk behaviors. The study is now examining how health changes over the course of early adulthood. [...]
The Add Health Study surveyed 90,000 7th to 12th graders, and has re-interviewed the same group of teens as they age. The study is made public to assist others studying adolescent health, and collects information on the following:
What kinds of topics does the study address?
The study collects information on:
*Physical and mental health, such as weight and height, injury and disability, dietary patterns and physical activity, substance use, access to and use of health care services, and suicide and depression
*Interpersonal relationships and sexual behaviors, such as family relationships, friendships, interracial relationships, faith community interactions, sexual activity, and sexual orientation
*Education, including cognitive ability and individual, family, peer, and community influences on school performance
*Delinquency and violence, including individual, family, peer, and community influences on delinquency and violence and risk factors for delinquency and violence
*Involvement in adult roles, including parenthood, jobs, marriage
*Genetic characteristics and biological measures that indicate the presence of specific diseases and disease processes
*Measures of the environments in which participants live and go to school
So this study provides a lot of data on the lives of teens. However, Kanazawa tries to pull information that wasn’t intended to be studied from the report, with no further discussion or references, and present it as fact. (In fact, would you know what the Add Health study was intended to do if we didn’t look it up?) Problematic, to say the least.
We had issues with Allure’s report on the changing face of beauty in the United States, but at least their methodology was much more clear – we knew how many people were surveyed, the images of the models they were shown, what questions they were asked, and how that compared to a similar survey done twenty years ago.
Interrogate the Author of the Study
Kanazawa calls himself “The Scientific Fundamentalist,” and claims to take “a Hard Look at the Truths of Human Nature.” His other articles include things like “Are All Women Essentially Prostitutes,” “Beautiful People Really ARE More Intelligent,” “What I Have Learned from Barry Goldwater,” and this statement on Eva Longoria and Tony Parker’s divorce:
Yes, I called it, nearly two years ago. I knew their marriage was very short-lived long before they themselves did. Once again, such is the power of the evolutionary psychological imagination. We know everything, not because we are special, but because we are evolutionary psychologists.
I’m a Mac, and I predict events before they happen.
I’m afraid to click the links for that rationale.
Amazingly, Kanazawa’s work fits neatly into this bingo card, created by the Punk Ass Blog:
Check for Scientific Racism
Wikipedia has a very useful summary (and a few interesting convos on the talk page) dealing with Scientific Racism. But the clearest example is actually found on the Wikipedia page for The Bell Curve, where an intrepid Wikipedian added a debunking guide for racist misapplications of science:
Evolutionary biologist Joseph L. Graves described the Bell Curve as an example of racist science, containing all the types of errors in the application of scientific method that have characterized the history of Scientific racism:
- claims that are not supported by the data given
- errors in calculation that invariably support the hypothesis
- no mention of data that contradicts the hypothesis
- no mention of theories and data that conflict with core assumptions
- bold policy recommendations that are consistent with those advocated by racists.
Be Wary of People Trying to Quantify What is Subjective
Beauty is in the eye of the beholder. And yet, every few years, someone tries to prove that x is definitively more attractive than y group. The closest science has been able to come to anything remotely resembling consensus is a link between symmetry and facial attractiveness.
Everything else is informed by personal preferences, how one interprets beauty, and cultural messages about beauty – which again, do change. What was beautiful in the 1980s and 1990s isn’t necessarily valued today. And globally, the idea of beauty shifts often. So trying to definitively state what is attractive and what is not is a bit of a losing game.
Remember that race is a social construct
Racebox.org shows how these alleged racial categories have changed over time. Here’s who you could be in 1890:
White People Swim, and Black People Run? Race, Science, and Athletics - Racialicious
Scientific Findings are not Public Service Announcements - Restructure
Interview with Joseph L. Graves - Addicted to Race
Guest Rant: Joseph L. Graves - Addicted to Race
James Watson’s Racism - Addicted to Race
(Image via PhD Comics, by Jorge Cham)
Thanks to readers Ruthi, Karen, and Lorenzo for sending in copies of the article!