How to handle an idiotic review

Image: Cover of The Idiot, by John Kendick Bangs (Harper and Bros. 1895). Yes, I know there’s a much more famous The Idiot.

If it hasn’t happened to you yet, it will: you submit a manuscript for publication and get back a peer review that makes you spit with fury.  “What an idiot!” you snarl, “how could a reviewer ever think that?”

Our peer review system works extremely well, overall*.  But reviewers are human, just like the rest of us – so a few of them just aren’t that good at it, and a few behave badly, and even the very best have bad days.  So you will get that idiotic review. What then?

Step one:  wait.  Do not write the quick, snappy response that you want to.  Instead, put the review away and don’t look at it again for at least 72 hours.  No matter how dispassionate we like to think we are, we’re human, and humans aren’t good at weighing the merits of criticism on first receipt.  (If you really must write an immediate response, write it outside your e-mail program, so you can’t hit ‘send’ by accident, and then delete it.)

Step two: ask yourself critically whether the review really is idiotic. It might be; but given that truly idiotic reviews are uncommon, you should seriously consider the possibility that you’re overreacting to some unfortunate phrasing cloaking a review that’s actually right.  All of us overreact this way. I overreact almost every time I read a review, and if I didn’t consciously watch myself for this behaviour, I’d dismiss every helpful criticism as “idiotic review” and my work would never improve.  Set yourself a high bar for the conclusion of idiocy.

Step three: decide what kind of idiotic you’re looking at.  There are four different ways a review can be idiotic:

(a) The reviewer mounts ad hominem attacks (that is, criticizes you rather than the manuscript).

(b) The reviewer makes comments that suggest a conflict of interest.

(c) The reviewer makes criticisms that suggest they didn’t read the manuscript carefully.

(d) The reviewer makes criticisms that are factually incorrect.

These four kinds of idiocy merit different responses, so you have to be clear which you’re dealing with.  Don’t rely only on your own judgement.  Discuss the review with a colleague who isn’t involved with the manuscript and who can be brutally honest with you when appropriate.

Step four: mount an appropriate response**.

(a) The ad hominem review. Personal attacks in a review are unprofessional*** (as well as unproductive), and you should never see them because an editor should redact them from the review before sending it on.  Should, but may not.  If you do get an idiotic review of this sort, your Response to reviews document should ignore the attacks.  It should, though, answer all the substantive criticisms that accompany the ad hominem ones – the latter don’t earn you a free pass from the former.  You can (and I’d argue should) send a separate note to the editor, along the following lines:

Reviewer #2 leveled a number of ad hominem attacks [examples].  In my Response to Reviews I ignore them, since they aren’t constructive.  I bring them up here, though, because they also aren’t professional, and I’m surprised that they weren’t redacted from the review before it was sent to me.  I’ve got a thick skin and won’t let comments like these discourage me; but I’d worry about a more easily discouraged author, perhaps making their first submission, thinking comments like these were typical of the publication process.  Can I suggest that future reviews from this reviewer be checked carefully before being sent on?

The point of doing this isn’t to get your own manuscript accepted – and it likely won’t.  But you should do it anyway as a contribution to improving our culture and our review process.  If you’re a grad student or other early-career author, it may be appropriate and more comfortable to have someone like a supervisor write and send this message in your stead.

(b) The conflict-of-interest review.  This is the rarest type (unless you count “please cite these 12 unrelated papers by Jones, but I’m not signing my review so you won’t know that I’m Jones” – very common, and indeed a conflict of interest, but not one worth worrying about).  It’s also the hardest to be sure of: most often, a review suggesting conflict of interest will be unsigned, and people’s attempts to guess the identities of anonymous reviewers mostly misfire.  But if you’re confident enough to do something, a conflict-of-interest review should be handled more or less the same way as an ad hominem review.

(c) The didn’t-read-it review.  You’ll be tempted to complain to the editor about the reviewer not doing their job.  Don’t, for two reasons.  First, such complaints (even if justified) won’t endear you to the editor or (should they see your revision) to the reviewer.  Second, complaining won’t improve your paper – but a little reflection on why the reviewer didn’t read thoroughly just might.  After all, reviewers should (and most do) approach a manuscript with the intent to read carefully; if they didn’t, perhaps your manuscript is less compelling than you’d like.  If you’re losing reviewers, you’ll definitely lose readers!  So think about what the reviewer didn’t read (or understand or remember), and ask why. Did your organization hide important material? Did your turgid writing make eyes glaze over? Did you test your reader’s patience with three paragraphs where one would have done?  Then sprinkle your Response to Reviews with bits like “Reviewer #2 thought X was a problem.  Actually, I had explained why X is correct (original MS lines xxx-yyy), but some poor organization made it easy for a reader to miss. I’ve revised to make this more apparent”. Actually, even if you don’t really think it was your fault, you’ve got nothing to lose and much to gain by pretending that you do.

(d) The factually-wrong review:  This is the easiest to deal with, but don’t let that ease make you cavalier.  Your Response to Reviews should explain clearly and dispassionately what the reviewer has gotten wrong, ideally backed up with citations. This is not a time to scold.  Sure, the reviewer “should know” that parametric ANOVA is robust to violations of the normality assumption (for example).  But saying so will only antagonize someone who could otherwise be your ally.  Besides – do you want to restrict the audience for your paper to people who know all the things you think they should?  Or would you rather let the reviewer’s mistake help you reach others who go similarly astray?  If the latter, then a good response is along the lines of “Reviewer #2 suggests that my interpretation doesn’t hold because Y.  Actually, Y is incorrect (brief explanation and citation); but because I suspect other readers may have the same worry, I’ve revised to address the issue at line zzz)”.  You may think it’s shockingly unlikely that any other reader could have such a foolish misunderstanding.  But you’re probably wrong, and even if you’re right, there is nothing to be gained by pointing it out.

All this advice has a couple of common threads.  First: reviewers and editors are human.  They make mistakes and have failings just like all of us.  Second: with the right attitude to revision, the idiotic review is an opportunity. Reviewers, in all their humanity, capture possible reactions of real readers – who are, after all, human too.  Even an idiotic review helps, because it can show you how to reach even those readers who might read you carelessly, or who have imperfect knowledge of your field.  They matter too.

For more about handling reviews (and a lot else), see The Scientist’s Guide to Writing

© Stephen Heard (sheard@unb.ca) May 2, 2016

Related posts:

• Are peer reviewers crazy, or are they saints?
• Is everything “broken”?
• Should you appeal when a journal rejects your paper?
• The dumbest thing I ever said to a reviewer

---

*^Yes, people frequently claim that “peer review is broken”.  It isn’t, and neither is all the other stuff people say is broken.  It’s just far easier, and grabs far more eyeballs, to write about how horrible everything is than to write about how most of the things we do as scientists work pretty well most of the time.

**^Assuming, that is, that you will be submitting a revision.  Which brings up this question: what if your manuscript is rejected without the option to resubmit, and the basis for this decision is an idiotic review?  You have the option of appealing that decision; but most of the time, I don’t think doing so is worth it.

***^As they are anywhere else in science, which is why this post is called “How to handle an idiotic review”, not “How to handle an idiotic reviewer”.

Advertisement