Image: Author expectations for “optimal” peer review: Figure 1 from Nguyen et al. (2015) PLoS ONE 10(8):e0132557.
Two things I saw last week motivated todays’ post. The first was Amy Parachnowitsch’s interesting blog post, wondering if peer review might sometimes be faster than she’d like: too fast for her to get head-clearing perspective by putting a manuscript away for a while. The second was a paper by Nguyen et al. reporting author opinions of how long peer review should take. Some of those opinions are absolutely astonishing.
I’ll get my astonishment in a moment, but first: how long should peer review take? I’m going to assume here that we all understand that peer review has two functions: a gatekeeping function (passing judgement on whether a paper should be published at all) and a manuscript-improvement function (providing comments that make the published paper better than the submitted manuscript). I’m going to assert that despite its occasional warts, the peer review process overall performs both these functions very well, and that we want it to continue performing both functions. So what kind of time should it take to get a manuscript through peer review?
I’m going to take a typical journal’s workflow, and guesstimate the minimum reasonable times we ought to allow for each step in the process*. This is not to say a given step couldn’t happen faster on occasion, but my guesstimates are such that I think those occasions will be rare. Thus, it would be patently unreasonable to expect anything faster while still allowing peer review to perform its functions. Here goes. Upload your manuscript, punch “submit”, and then allow:
1 business day** for the journal office to process your submission (format checks, etc.)
- 3 for the Editor-in-Chief (EiC) to do a quick read and assign to an associate editor (AE)
- 3 for the AE to read the manuscript, decide if it should go to review, and invite reviewers
- 7 for the journal office to contact reviewers and for enough to agree to review
- 10 for the reviewers to complete their reviews
- 1 for the journal office to process the reviews and notify the AE
- 5 for the AE to consider the reviews, read the MS more thoroughly, and write their own review and decision recommendation
- 4 for the EiC to read the MS, consider the AE’s recommendation, and make a final decision
- 1 for the journal office to finalize the paperwork and issue the decision letter
This comes to 35 business days, or a bit more than 7 weeks (depending on the luck of the statutory-holiday draw). If some of the individual times seem long, remember that everyone except possibly the journal office staff is unpaid and doing their journal work on top of all their other responsibilities. Sure, I can (sometimes) review a manuscript in half a day; but when I receive one to review, it goes in the queue with everything else on my plate. There is, after all, no reason the review I was asked for this morning is more important than the six thesis chapters my students sent me last week.
So 7 weeks is, I think, the absolute minimum that it’s reasonable to expect – not the average, but the minimum. And while online technology has drastically cut peer-review times over the last decade or two (when I was a grad student, 16 weeks was about the best you could hope for), I don’t think any amount of technology will cut them any further. The constraints now aren’t systems-related; they’re process-related. The process, I think, has to take about that long if we want it to work. This means, by the way, that journals that compete on claims of peer-review turnaround faster than 7 weeks are either promising something they can’t deliver, or abandoning the quality that comes with serious peer review. (Enhancements like transferability of peer reviews may cut time for subsequent consideration of the same manuscript, but I don’t see potential for major cuts in the first go-around. And we hear occasional calls for a complete reimagining of the whole journal-publication system, but I have yet to see a convincing case for this.)
So, peer review is unlikely to ever be faster than we’d like (you’re welcome, Amy). Is it slower than we’d like? Here’s where the Nguyen et al. paper comes in; prepare to be as astonished as I was. Nguyen et al. surveyed authors in conservation biology***, asking them questions about typical peer-review times (defined as everything from submission to decision, as in my breakdown above), and about what they considered short, optimal, and long review times. The results blew me away.
What do authors want? Their average “optimal” review time (Nguyen et al.’s Figure 1, reproduced above) was six weeks – a week less than the 7 weeks I think is the minimum reasonable time. Five respondents thought one week was optimal (and, presumably, each of them also wanted a pony).
What do authors consider to be “slow” peer review? On average, 14 weeks – but a significant number of authors have stricter standards. Fifty (a full 10% of respondents) thought a “slow” review meant 7 weeks or less! One respondent said one week was a “slow” review – demonstrating beyond reasonable doubt either that some people like to punk survey studies, or that some people are remarkably naïve****. The non-trivial number of votes for 2 and 3 weeks suggests, unfortunately, the latter.
I just can’t wrap my head around these expectations. Look: peer review has a job to do, and doing that job takes time. It would take time even if the whole process was handled by full-time, paid professionals; but it’s largely handled by volunteers who are also busy with other things. Peer review that takes a couple of months is not evidence of a broken system, and isn’t a reason we should be looking for alternatives. I, of course, believe that each and every one of my papers is an important advance for science (grin), but I think the world can generally wait a couple of months while peer review makes them better.
There is, it’s true, pressure on the peer-review system, with submission volumes rising continuously and faster than the pool of available reviewers (and AEs). Unreasonable expectations about the speed of the process make this pressure worse. An obvious way to alleviate that pressure is to take everyone who thinks a 7-week peer review process is “slow”, and invite them to review an extra manuscript each month – or to sign on as an AE. We can use the help.
UPDATE: See a closely related post from Terry McGlynn, who independently comes to a quite similar analysis.
UPDATE#2: For more grumpiness on the subject, this time aimed at journal rather than authors, see Things that are more important to me than reviewing your manuscript.
© Stephen Heard (firstname.lastname@example.org) August 31, 2015
*^I think my guesstimates are pretty good for the steps I have experience with (reviewer and associate editor steps). They’re probably underestimates for the steps I know less about (journal office and editor-in-chief); at least, I always assume I’m underestimating how hard somebody else’s job is.
**^Yes, I frequently work weekends and holidays, including handling and reviewing manuscript; but I’m not proud of it. And you certainly don’t get to assume that I’m going to work a holiday to handle your manuscript!
***^Whether other authors would have given similar answers is unknown; but based strictly on anecdote, I’d expect lots of fields to be even less patient.
****^“Naïve” was not the first word that sprung to mind, but I’ll give a little benefit of the doubt.