A different way of evaluating research
PRACTICAL GUIDE 11
Open Science represents a profound change in science and research. At the core it always includes the public transparency of decisions. This also applies to evaluation practices that until now took place between individual and mostly anonymous reviewers and authors.
Rethinking peer review
Peer review is a central reviewing process in research. The result of such a review is good well-founded scientific research. Usually, the publishers or the journals organise the process. They submit the manuscript to other researchers in the same field. However, the system has a few weaknesses, in particular because reviewers can be competitors of the author. Sometimes reviewers work on research topics which are not a close match to their own expertise.
And the peer review mechanism cannot protect against scientifically doubtful publications. Since 2000, diverse fraudulent or questionable articles have come to light which were based on data manipulation or plagiarism, but nonetheless passed the peer review process (Grieneisen & Zhang, 2012; Fang, Steen & Casadevall, 2013).
Open Science has given rise to Open Peer Review practices with two main modalities: the names of the reviewers are disclosed and/or the review takes place on a platform that allows all users to comment the article. This practice has been facilitated by preprint platforms such as arXiv and bioRxiv, which journals can use to collect comments.
Example
The Peer Community in platform organises the reviewing of scholarly manuscripts deposited in an open archive. After a determinate number of positive reviews the authors receive the results of the peer review process, so-called recommendations that are openly accessible and citable and that have a Digital Object Identifier. This enables journals to publish articles freely without requesting reviewers.
In some disciplines, for instance in economics, the pre-registration of hypotheses and registered reports in registers or journals has changed the peer review practice. Peer review takes place in two stages which reduces the effect of publication bias (the tendency to publish only positive findings) and emphasises the research process.
Reading tip: “What the Open Science movement can learn from the Covid-19 crisis” – Interview with Professor Isabella Peters, on Open Peer Review.
Good luck with your research!
Date: March 2021
Questions, comments and notes are welcome at open-science@zbw.eu