Alternative Peer Review

This isn’t right, this isn’t even wrong.
Wolfgang Pauli (1900-1958), upon reading a young physicist’s paper

Struggling with peer reviews (including some really mean comments), I thought about alternative solutions. Don’t get me wrong, I think that peer review, the examination of your scientific work by selected peers before it gets published (if ever) in a scientific journal, is crucial. But I am not so sure that the reviewers chosen by the editor are always the best choices.

I can understand that finding reviewers is a difficult job. Good (even bad) reviewing costs time, effort, and in that time you cannot write your own papers. Some work you review might hit too close to home to make you comfortable, and given the amount of harmful reviews a scientist has received in his career, some like to invoke the norm of reciprocity. And I think most editors can’t be choosy regarding their reviewers, given how hard it is to find a person who is expert on the same topic as the submission, but not so close to be in direct competition.

So, perhaps, instead of using specific reviewers as gate keepers to the journal, a dedicated service where interested readers request submitted articles that deal with their research topics would be more helpful. They receive them for free — with the condition that they rate the articles regarding their methodological quality and whether it advances the field (as minimum) and give other comments (optional). Kinda like a user poll, the information is aggregated and anonymized and made public for other readers. Harmful comments are reprimanded by other readers, including the author (the comments are anonymous but each reviewer is accountable given that the computer can match the contributions to individuals). Readers can also ask the authors to give more information in areas where the article is unclear and they are informed about these changes and can change their decision regarding the article. After a few of these feedback loops (like a Delphi study, done anonymously, but where one is able to address individual reviewers), the article is either accepted in the body of the journal articles (and made publicly available for everyone, not only the subscribing authors) and the journal guarantees for the quality of the article, or not.

In summary, the interested persons of the field would judge the articles, not two or three representatives chosen by the editor which may or may not represent the field and may or may not be biased. Both reviewers and authors profit by this system and due to the open discussion even of unpublished work it might advance the field more than traditional peer review.