Inspiring STEM Consulting has recently worked with a number of clients to research the open peer review market for scientific journals. Specifically, we have looked at whether open peer review is a sustainable and scalable model and also considered the practical issues.
Among scientific publications, it is still very clear that although open peer review is embedded in the ‘open science’ DNA, it remains an emerging, evolving practice, especially when associated with the publication of referee reports.
However, scientific journal peer review faces multiple challenges, for example predatory journals, publication fraud and the effort for greater and more robust research integrity. I strongly believe that transparency and openness of peer reviewing lies at the heart of the solution.
What is generally defined as ‘open peer review’ takes the form of ‘open identities’, a type of peer review according to which reviewers’ names are disclosed. On acceptance and publication of the paper, the associated referee report may, or may not, be published.
We are now also witnessing the evolution of a publishing ecosystem with data, and not the article, at the epicentre. This raises the new challenge of peer reviewing data and ensuring that these are F.A.I.R – findable, accessible, interoperable and reusable.
Scholarly peer review remains a subjective process of subjecting an author’s scholarly research to the scrutiny of experts in the same field. While there are indications that more transparent forms of peer review are gaining acceptance, the single-blind and double-blind peer review processes still dominate.
In 2017, a cross-disciplinary survey1 on open peer review for the OpenAIRE2020 project reported on attitudes and experiences among editors, authors and reviewers. Results showed that 60% of respondents believed that open peer review should be mainstream scholarly practice, but attitudes to individual traits varied, and open identities were not generally favoured. This compares well to the 2016 Publishing Research Consortium Peer Review Survey,2 according to which 59% of authors favoured open reports but only 31% open identities.
With both surveys, and despite the clear benefits, the underlying fear is that open identities could cause reviewers to weaken their criticisms or could lead to retaliation from authors. What is encouraging is that OpenAIRE reported high levels of experience, with 76% participating in open peer review either as an author, reviewer or editor. Additionally, there were high levels of support for open reports and final-version commenting, but again, respondents were against opening reviewer identities to authors, with 50% believing it would make peer review worse.
There remains a gulf between attitudes and practice, with less than 2% of scientific journals allowing the publication of referee reports. Among these are some prestigious titles, including the BMJ, Nature Communications and F1000 Research.
Another key finding of a Publons survey3 is the striking difference across age groups. Over 40% of under 26-year olds are likely, or highly likely, to review for open peer review journals, compared with 22% aged 56–65 years. We can make a fair assumption that at least the appetite for open peer review will increase over the next generation.
The potential benefits of publishing referee reports seem obvious. However, the problem remains that generally, only editors, authors and (sometimes) reviewers see referee reports. This enables several forms of abuse: referees might be superficial, rude or biased; authors might respond inadequately to reasonable criticism; editors might not hold authors or reviewers to account; and predatory publishers can charge fees without providing quality review.
Studies of published peer reviews are small, making effects hard to ascertain. Nonetheless, the evidence so far suggests that the scientific community finds published reports valuable. An Elsevier pilot studyfound that one-third of its website visitors accessed peer review reports, and several editors said they used published reports as instructive examples for inexperienced reviewers.
Also, editors at the European Journal of Neuroscience, which launched transparent review at the end of 2016, report that referees are writing better reviews and returning them more promptly.
Some disciplines are keener than others. Nature Communications found4 that, given a choice, authors (and reviewers) of more than 70% of its evolution and ecology submissions opted for published reports. The percentage was lower than 50% for submissions in physics.
Overall, open peer review remains an evolving phenomenon that requires continued monitoring through surveys. However, it feels very much right now that, despite experimentation by a few select publishers, the revolution has yet to happen.
So where might the real push come from? Well, as with open access publication, I believe that once the science funders are on board, the majority will begin to not just listen intently but make plans for real implementation.
So, in this respect, the recent announcements recommending publishing referee reports by the Wellcome Trust, the Howard Hughes Medical Institute and ASAPbio, a non-profit organization that encourages innovation in life sciences publishing, demand attention. Let’s see whether these recommendations transition to enforced mandates.
References
- Ross-Hellauer T, Deppe A, Schmidt B. Survey on open peer review: attitudes and experience amongst editors, authors and reviewers. PLOS ONE 2017;12:e0189311.
- Publishing Research Consortium. PRC Peer Review Survey 2015 (Mark Ware Consulting, 2016).
- Peer reviewers unmasked: largest global survey reveals trends. Nature News. 7 September 2018.
- Transparent peer review one year on. Nat Commun 2016;7;13626.
