A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.
Max Planck
I just noticed that I apparently haven’t recommended a paper by Trafimow and Rice (2009) yet. Published in “Perspectives on Psychological Science” the authors ask the question “What If Social Scientists Had Reviewed Great Scientific Works of the Past?” and the answer isn’t all that … “big on dignity” (to quote Doctor Who) for social scientists: If psychologists had reviewed the great works of science in the past, many of the great discoveries wouldn’t have made it into the “established journals”.
The reasons they give for this are well-worth reading — packed in a great deal of humor you start laughing (to read “Dear Albert”‘s (Einstein) Major Revision review or “Dear Isaac”‘s (Newton) Rejection review), until you cry, not necessarily because it’s funny. They pose some uncomfortable questions and challenge assumptions, for example:
“Few people would be willing to assert that progress in the behavioral sciences has been as impressive as the progress made in other sciences. Usually, when people discuss these matters, they present reasons to justify the differences: the other sciences have existed for longer; the behavioral sciences are more difficult because the mind is less tangible than the body, the world, or the universe; there is more funding for other sciences than for the behavioral sciences; and so on. Although some of these justifications may have some merit, there is another possibility that behavioral scientists rarely consider: perhaps they are not as effective in their scientific reasoning and in the way they evaluate scientific research.”
Trafimow & Rice (2009)
The authors discuss whether reviews in Psychology are overly strict and while they agree that we need a gatekeeper to make sure scientific work follows standards, reviewers in the social sciences often go to far:
Clearly, to some extent, this [“Gatekeeper” process] is appropriate, but as we have seen, it can be taken too far. An example of this is when reviewers note every case where an author breaks some sort of ‘‘rule,’’ duly notes it in his or her review, and then uses the violations as the reason for recommending rejection.
Trafimow & Rice (2009)
What really makes this article worthwhile reading is that it challenges your assumptions about peer-review, asks the right (albeit uncomfortable) questions, gives hints what to focus on, to make a differentiated judgment and give good reviews that improve science.
Highly recommended.
