The Myth of Self-Correcting Science - Sarah Estes - The Atlantic

The Myth of Self-Correcting Science

DEC 20 2012, 10:34 AM ET 5Recent academic scandals highlight a history of data falsification and questionable research in social psychology, and serve as calls to action. 

Carlos Jasso/Reuters

Over the last two years, the field of psychology has endured a wave of scandal bookended by fraud cases involving Harvard primatologist Marc Hauser and Dutch social psychologist Diederik Stapel. Even researchers desensitized by scandal-fatigue did a double take when the final report on Stapel's case came out last month. The extent of his creative misinterpretation of the facts make the Hauser case look like child's play. Stapel not only manipulated and fabricated data, he invented entire schools where said data was allegedly collected.

Because science has no real I-O police for research then this Iv-b and V-Bi disconnect can grow quite large, the penalties might be low such as losing a job or research grant.

As if the fraud files weren't enough, then come the mea culpas -- salt in the wounds for students and colleagues still recovering from shattered reputations and a shaken faith in science. The two men released two very different statements telling very similar stories of reckless, ruthless ambition and playing the odds against getting caught. Stapel's "narcissistic wail" was so emotional and contrite as to seem a bit unhinged, while Hauser's read as a cold, calculating non-admission of guilt.
Hauser deftly concedes chagrin for errors made within his lab "whether responsible for them or not," implying that the same students bullied into committing academic fraud were somehow responsible for the car veering off the cliff. Stapel faults a noxious combination of publication pressures, addictive tendencies, and assorted personality issues for his downfall. And while publication pressure was among those issues, he caps off his mea culpa with a plug for his new book -- Derailment, a collection of his therapeutic diaries.

Scientists can become Iv agents in effect committing fraud for the commission of higher wages, research grants, jobs, etc. These are considered to be errors from normal practice, the normal center of V-Bi. This abnormal behavior can reach a tipping point or Iv-B ceiling and go off the cliff into free fall where those fastest at distancing themselves from the problem while profiting at the time do best.

The Slippery Slope

It's easy to revel in the high drama surrounding the downfall of a Hauser or Stapel, but what about the journals that published these scholars? Stapel was a widely cited and highly revered figure. His fraud went undetected for decades in spite of eerily perfect data sets and improbable statistical values. According to Tilburg University's final report, Flawed Science, "There was a general neglect of fundamental scientific standards and methodological requirements from top to bottom."

When journals become agents for V companies and universities rather than neutral I then chaos can grow secretly until it hits a ceiling and crashes.

Scientists fought back, noting that it is rare for reviewers in any field to detect fraud and demanding an apology for the 'slanderous conclusions' drawn in the report.

Without I-O police being available for whistle blowers there is little incentive for scientists to moderate this chaos before it collapses. Instead they try to profit from it and get out before it hits the ceiling.

Social psychologist Kate Ratliff,teaching at Tilburg when the scandal broke noted, " It's a small community and people considered Diederik a friend and mentor...No one understands why these young researchers didn't realize that it was weird that Diederik was giving them datasets. But you learn from watching others. And if there are no others, how would you know what's weird or not? I think that people started out being really sympathetic toward them and have gotten more and more punitive as time passes and hindsight bias kicks in. I think that's really, really unfair."

In Iv-B people think tactically and compete with each other in a secretive environment, if others are doing something then it seems ok and the objective is to beat them. This makes the problem grow exponentially until it hits a ceiling.

They managed to find statistically significant evidence for the absurd hypothesis that listening to a Beatles song could make you 1.5 years younger

Almost more alarming than the few individuals committing academic fraud are the high percentage of researchers who admitted to more common questionable research practices, like post-hoc theorizing and data-fishing (sometimes referred to as p-hacking), in a recent study led by Leslie John.
For the uninitiated: post-hoc theorizing involves creating or revising a hypothesis after you've collected the data; data-fishing entails running a study, continually checking the data after each participant, and stopping as soon as you see a significant result. These practices are eschewed by some, but plenty of others embrace them. Joseph Simmons and colleagues ran a simulation showing how unacceptably easy it was to attain statistical significance using these 'degrees of researcher freedom.' By employing four of these questionable practices at once, they managed to find statistically significant evidence for the absurd hypothesis that listening to a Beatles song could make you 1.5 years younger.

V-Bi statistics can be affected by chaos as the vertical part of Pascal's Triangle, Iv-B researchers can manipulate data chaotically so it is no longer random.