Flawed Science — Or: Where do we go from here?

“Before you go any further, let me remind you things were very chaotic back then. Now we don’t have the problems we had back then. We all did what we had to do. And in those circumstances, we did the best we could. That’s all I have to say about that.”
V for Vendetta

So, the final report about the data falsification (PDF) done by (former) social psychologist Diederik Stapel is available (website for the report). For those who are not working in psychology or are not interested in bad science — a renowned social scientist, a ‘Golden Boy’ turned ‘exceptional researcher’, had falsified and fabricated data for many years. And I really mean fabricated — to quote the report:

“The experiments were seemingly executed under the complete supervision of Mr Stapel alone. According to his own account, Mr Stapel had excellent contacts with many educational institutions in the country. They were apparently always willing to be persuaded by him to perform studies of this kind, sometimes with the assistance of (fictitious) paid research assistants. To compensate the schools for their efforts, Mr Stapel claimed to give lectures and provide the schools concerned from time to time with computers and video projectors. It is apparent from Mr Stapel’s statements to the Levelt Committee that the questionnaires and additional materials loaded into the back of his car (e.g. bars of chocolate for use in an experiment) never went to schools but were dumped in a container [my emphasis]. Mr Stapel himself would then sit at his computer with an empty questionnaire to create a corresponding dataset. He gave his partners (PhD students, co-authors and colleagues) the impression that data were collected at the schools by (unknown) assistants, who also processed and coded the data. The data ‘obtained’ in this way were then given directly to Mr Stapel: never to the partners. All these ‘efforts’ usually resulted several weeks later in a (fictitious) dataset being made available in its entirety to the partner for more detailed analysis, or even directly in the form of tables with the necessary averages, standard deviations, reliability checks, test results, and so on. The partner would be able to start writing the article immediately, on his own or in intensive cooperation with Mr Stapel.”
Quote from “Flawed science: The fraudulent research practices of social psychologist Diederik Stapel” by the Levelt, Noort, and Drenth Committee

When it became public — due to three courageous whistle-blowers (his own doctoral students) — it lead to 30+ retracted papers and much pain on every side involved (and frankly, I don’t even want to imagine how Mr. Stapel, his PhD students, or the scientists attempting to replicate his (fictitious) results feel).

The report (PDF) is very interesting … because it now only shows how he falsified the data for his (and his PhD students’) publications, but also how the research culture made this fraud possible. Because let’s face it — science is … great, but the working conditions in science are not. Publish or perish is a terrible way to do science — sure, science should get results, but popularity — which you need to get through 2-3 peer-reviewers — interferes with ‘truth’ (dangerous word, I know). To be successful in science, you need to act like a politician and play the game by its own rules — to quote Josh from “The West Wing”: “We’re game players, not rule makers.” — you need to publish at (almost) any cost and must not invest the time to think about the system you are working in.

But where does this system leave us — scientists and the general public alike?

Frankly, I think that a lot of scientists — at least in psychology — are manipulating their data. Perhaps not to the extend of Mr. Stapel, but a data point here, a dropped study there, it starts small. Perhaps the reviewer ‘demanded’ clearer results, perhaps you had ‘good’ reasons. But in the end, science gets distorted, it loses its key strength: Getting closer to the ‘truth’ over time due to its self-correcting nature. Because how can we get closer to the truth if we manipulate data — not matter how small — and do not publish failed experiments? If there are enough people doing experiments on a flawed theory, some will get statistically significant results, even without manipulation.

It’s a complicated issue, and yes, I think when he put questionnaires into his car and dumped them, it should have been time to question what he was doing. But it is easy to condemn. I think that psychology has given us incredible, counter-intuitive insights — about human nature (just think about Milgram, not even trained therapists were able to foresee the results), about the conditions in which people help each other (or don’t — and it has nothing to do with ‘bad’ human beings), conformity, why it matters who a person you like likes (unfortunately), and much more. But I also think that the system is stacked against good research. To survive you must deliver immediate results. Some research institutions even increase the pressure by giving out short term contracts (e.g., 1 or even 1/4 year contracts for young researchers) and make any extension of the contract dependent on their (short term!) success. Very motivational … seriously, that’s the quickest way to scorch the earth for the next generation of scientists.

Because how can you come up with good research under these conditions? Even more when you consider that many professors and post-docs are mainly concerned with their own careers (and at least post-docs have to!) and have no time to give good advisory.

Like written, the report (PDF) is worth reading — but it’s only the tip of the iceberg. Science — at least in psychology — has to decide where to go from here. Whether we continue with a system that favors the lucky, conservative, or manipulative researchers, or whether we rethink the way we are doing science.

Update

One thing I have forgotten: I do not believe that the system will correct itself soon. Problem is — all people who are in positions of power in science (i.e., professors) have gotten to their positions in that system. They have already paid the price and now want to reap the promised rewards. So why would they act against their interest. I fear that the system first has to crash totally — hit rock bottom — before being rebuild in a way that works (better).

Update 2 (2013-07-01)

According to a Nature Blog Posting, Stapel has been convicted to 120 hours of community service, as the about 2.2 Million Euro research money was not used for personal gain and he does not claim 18 months of half-pay salary. Still, if you consider the scope of the scientific misconduct, I think that setting his hourly wage for the community service of 18333€/hour is perhaps a bit high.

Update (2014-06-08)

Reading Update 2 again — wait, not for personal gain? He used it to foster his career, to get a name and reputation in science. How is that not for personal gain?

13 Trackbacks / Pingbacks

  1. Negative Motivation | ORGANIZING CREATIVITY
  2. Two blogs to subscribe to if you are interested in Science | ORGANIZING CREATIVITY
  3. Singapore Statement on Research Integrity | ORGANIZING CREATIVITY
  4. Workshop: Scientific Work — Colleagues & Networking | ORGANIZING CREATIVITY
  5. Workshop: Scientific Work — Dealing with Data | ORGANIZING CREATIVITY
  6. Making sense of scientific news stories | ORGANIZING CREATIVITY
  7. What Science Should be About (but Academia Often Isn’t): The Joy of Discovery | ORGANIZING CREATIVITY
  8. Short Series about Science: #4 Scientific Responsibility | ORGANIZING CREATIVITY
  9. A statue has never been erected in honor of a critic. Sleuths howerver … | ORGANIZING CREATIVITY
  10. A statue has never been erected in honor of a critic. Sleuths however … | ORGANIZING CREATIVITY
  11. In Science We Trust, In Scientists We Shouldn’t. | ORGANIZING CREATIVITY
  12. Science, Fraud, and the Flow of Data | ORGANIZING CREATIVITY
  13. Using the Fraud Triangle to Explain Scientific Misconduct | ORGANIZING CREATIVITY

Comments are closed.