How to handle an idiotic review

Image: Cover of The Idiot, by John Kendick Bangs (Harper and Bros. 1895). Yes, I know there’s a much more famous The Idiot.

If it hasn’t happened to you yet, it will: you submit a manuscript for publication and get back a peer review that makes you spit with fury.  “What an idiot!” you snarl, “how could a reviewer ever think that?”

Our peer review system works extremely well, overall*.  But reviewers are human, just like the rest of us – so a few of them just aren’t that good at it, and a few behave badly, and even the very best have bad days.  So you will get that idiotic review. What then?

Step one:  wait.  Do not write the quick, snappy response that you want to.  Instead, put the review away and don’t look at it again for at least 72 hours.  No matter how dispassionate we like to think we are, we’re human, and humans aren’t good at weighing the merits of criticism on first receipt.  (If you really must write an immediate response, write it outside your e-mail program, so you can’t hit ‘send’ by accident, and then delete it.)

Step two: ask yourself critically whether the review really is idiotic. It might be; but given that truly idiotic reviews are uncommon, you should seriously consider the possibility that you’re overreacting to some unfortunate phrasing cloaking a review that’s actually right.  All of us overreact this way. I overreact almost every time I read a review, and if I didn’t consciously watch myself for this behaviour, I’d dismiss every helpful criticism as “idiotic review” and my work would never improve.  Set yourself a high bar for the conclusion of idiocy.

Step three: decide what kind of idiotic you’re looking at.  There are four different ways a review can be idiotic:

(a) The reviewer mounts ad hominem attacks (that is, criticizes you rather than the manuscript).

(b) The reviewer makes comments that suggest a conflict of interest.

(c) The reviewer makes criticisms that suggest they didn’t read the manuscript carefully.

(d) The reviewer makes criticisms that are factually incorrect.

These four kinds of idiocy merit different responses, so you have to be clear which you’re dealing with.  Don’t rely only on your own judgement.  Discuss the review with a colleague who isn’t involved with the manuscript and who can be brutally honest with you when appropriate.

Step four: mount an appropriate response**.

(a) The ad hominem review. Personal attacks in a review are unprofessional*** (as well as unproductive), and you should never see them because an editor should redact them from the review before sending it on.  Should, but may not.  If you do get an idiotic review of this sort, your Response to reviews document should ignore the attacks.  It should, though, answer all the substantive criticisms that accompany the ad hominem ones – the latter don’t earn you a free pass from the former.  You can (and I’d argue should) send a separate note to the editor, along the following lines:

Reviewer #2 leveled a number of ad hominem attacks [examples].  In my Response to Reviews I ignore them, since they aren’t constructive.  I bring them up here, though, because they also aren’t professional, and I’m surprised that they weren’t redacted from the review before it was sent to me.  I’ve got a thick skin and won’t let comments like these discourage me; but I’d worry about a more easily discouraged author, perhaps making their first submission, thinking comments like these were typical of the publication process.  Can I suggest that future reviews from this reviewer be checked carefully before being sent on?

The point of doing this isn’t to get your own manuscript accepted – and it likely won’t.  But you should do it anyway as a contribution to improving our culture and our review process.  If you’re a grad student or other early-career author, it may be appropriate and more comfortable to have someone like a supervisor write and send this message in your stead.

(b) The conflict-of-interest review This is the rarest type (unless you count “please cite these 12 unrelated papers by Jones, but I’m not signing my review so you won’t know that I’m Jones” – very common, and indeed a conflict of interest, but not one worth worrying about).  It’s also the hardest to be sure of: most often, a review suggesting conflict of interest will be unsigned, and people’s attempts to guess the identities of anonymous reviewers mostly misfire.  But if you’re confident enough to do something, a conflict-of-interest review should be handled more or less the same way as an ad hominem review.

(c) The didn’t-read-it review.  You’ll be tempted to complain to the editor about the reviewer not doing their job.  Don’t, for two reasons.  First, such complaints (even if justified) won’t endear you to the editor or (should they see your revision) to the reviewer.  Second, complaining won’t improve your paper – but a little reflection on why the reviewer didn’t read thoroughly just might.  After all, reviewers should (and most do) approach a manuscript with the intent to read carefully; if they didn’t, perhaps your manuscript is less compelling than you’d like.  If you’re losing reviewers, you’ll definitely lose readers!  So think about what the reviewer didn’t read (or understand or remember), and ask why. Did your organization hide important material? Did your turgid writing make eyes glaze over? Did you test your reader’s patience with three paragraphs where one would have done?  Then sprinkle your Response to Reviews with bits like “Reviewer #2 thought X was a problem.  Actually, I had explained why X is correct (original MS lines xxx-yyy), but some poor organization made it easy for a reader to miss. I’ve revised to make this more apparent”. Actually, even if you don’t really think it was your fault, you’ve got nothing to lose and much to gain by pretending that you do.

(d) The factually-wrong review This is the easiest to deal with, but don’t let that ease make you cavalier.  Your Response to Reviews should explain clearly and dispassionately what the reviewer has gotten wrong, ideally backed up with citations. This is not a time to scold.  Sure, the reviewer “should know” that parametric ANOVA is robust to violations of the normality assumption (for example).  But saying so will only antagonize someone who could otherwise be your ally.  Besides – do you want to restrict the audience for your paper to people who know all the things you think they should?  Or would you rather let the reviewer’s mistake help you reach others who go similarly astray?  If the latter, then a good response is along the lines of “Reviewer #2 suggests that my interpretation doesn’t hold because Y.  Actually, Y is incorrect (brief explanation and citation); but because I suspect other readers may have the same worry, I’ve revised to address the issue at line zzz)”.  You may think it’s shockingly unlikely that any other reader could have such a foolish misunderstanding.  But you’re probably wrong, and even if you’re right, there is nothing to be gained by pointing it out.

All this advice has a couple of common threads.  First: reviewers and editors are human.  They make mistakes and have failings just like all of us.  Second: with the right attitude to revision, the idiotic review is an opportunity. Reviewers, in all their humanity, capture possible reactions of real readers – who are, after all, human too.  Even an idiotic review helps, because it can show you how to reach even those readers who might read you carelessly, or who have imperfect knowledge of your field.  They matter too.

For more about handling reviews (and a lot else), see The Scientist’s Guide to Writing

© Stephen Heard (sheard@unb.ca) May 2, 2016

Related posts:


*^Yes, people frequently claim that “peer review is broken”.  It isn’t, and neither is all the other stuff people say is broken.  It’s just far easier, and grabs far more eyeballs, to write about how horrible everything is than to write about how most of the things we do as scientists work pretty well most of the time.

**^Assuming, that is, that you will be submitting a revision.  Which brings up this question: what if your manuscript is rejected without the option to resubmit, and the basis for this decision is an idiotic review?  You have the option of appealing that decision; but most of the time, I don’t think doing so is worth it.

***^As they are anywhere else in science, which is why this post is called “How to handle an idiotic review”, not “How to handle an idiotic reviewer”.

Advertisements

20 thoughts on “How to handle an idiotic review

  1. Margaret Kosmala

    “If you really must write an immediate response, write it outside your e-mail program, so you can’t hit ‘send’ by accident, and then delete it.”

    OR… write it outside your e-mail program and then save it to your Angry Folder. (Tip: don’t ever re-read anything in your Angry Folder unless you’re still angry about it.) I find that writing down my initial reaction helps me get past it quicker. And I don’t like to throw things out…

    Liked by 1 person

    Reply
      1. Margaret Kosmala

        A couple of reads into the Angry Folder will leave you going, “eee. I don’t like this person (former you). I don’t think I’m going to read any more of their writing…” So not too hard to stick to.

        Liked by 1 person

        Reply
  2. Miriam Richards

    I like the idea that you shouldn’t reread the reviews for 72 hours to give yourself time for a more reasoned perspective. I wish 3 days would do it! Recently I found myself fuming after waiting 3 months!

    Like

    Reply
  3. Chris MacQuarrie (@CMacQuar)

    Upon receiving reviews of my very 1st paper I, being keen and eager, wrote a quick and not-so-tactful response to the reviewer. I then sent this off to my MSc. supervisor who returned it to me, quite thoroughly edited with the message “You have written a good response, I have just made it nicer”. I try to remember that when I’m writing a response to reviewer. Its easy, and a lot more fun, to be a jerk when writing quickly and in the heat of the moment. It’s much harder to write tactfully and dispassionately.

    Liked by 1 person

    Reply
  4. Pingback: Recommended reads #77 | Small Pond Science

  5. viewsfromunderthebus

    As a just-graduated PhD, I received a “didn’t read it” review of my dissertation manuscript that the reviewer couldn’t even bother to compile into a written review. It was my submission with handwritten comments in the margins, clearly made on a first reading of the paper. This review took the prominent journal to which I submitted it four months. This is why I think you’re a bit too generous to reviewers and too conciliatory. Reviewers have power that authors do not and should take their responsibility very seriously.

    Like

    Reply
    1. ScientistSeesSquirrel Post author

      It’s certainly true that reviewers sometimes don’t do the job we’d wish. (Of course, we don’t pay them what they might wish for their time.) So yes, I agree that sometimes one gets a review that shouldn’t have been written! The question is what to do next, after you’ve gotten it, and that’s where I focused my post. If that sounds like being too generous to reviewers, that wasn’t really what I was thinking about! But I’d stand by my advice, because I’m not sure what being less conciliatory would accomplish. Thanks for your comment – and I hope your other review experiences have been better. Most of them are!

      Liked by 1 person

      Reply
  6. SB

    I think something that’s overlooked here is the “it was a perfectly good proposal but we don’t have enough money but can’t say that review”. I’ve had a few of these from NIH where they really couldn’t fault the research so went after silly things like my Institute’s RCR statement. Yes I was upset, and yes they leant on really flimsy reasoning not to give me the money and yes I think that the fact I was at a lower ranking university gave them grounds to discriminate but ultimately it came down to the fact that at that point they were only funding around 8% of applications, so to reject 92% of what comes through the door means finding some really ridiculous reasons to not fund.
    My coping strategy was to play “A Song for Phil Daoust” by Tim Minchin and sing along to it loudly and angrily in my car (it’s a very rude song about arts critics, but works for reviewers too).

    Like

    Reply
      1. David Taylor

        Good comment. It reminded me of my first NSF proposal, as a PhD student years ago applying for a dissertation research grant. The reviewers liked it, with one exception whose only critical comment was that I didn’t ask for enough money. My advisor went apeshlt, and told the program officer to just give me more money! I suspected, in hindsight, that they simply couldn’t fund all of the good proposals they received, and look to such ridiculous evaluations to justify not funding everyone who deserves it. Happily I’ve had more success with NIH over the years!

        Liked by 1 person

        Reply
  7. Barbara Piper

    I’m coming to this a little late, but wanted to endorse the suggestions… My own strangest experience was not so much a peer review as a first response from an editor who suggested an entirely different approach for the paper I submitted. I thought that editor had missed the point completely, but since he was a professional friend I decided to respond simply that I appreciated the advice, and would revise the paper to make my perspective more clear. I then sent the paper in exactly the same form to another journal, which accepted it without revisions. When it finally appeared in print — word-for-word exactly as I had submitted it to the original editor — that editor friend commented that if I had send him that draft he would have accepted it without hesitation! I left it alone, in the interests of professional collegiality, but it reminded me that even the best editors can slip occasionally….

    Liked by 1 person

    Reply
  8. Pingback: Weekend reads: Idiotic reviews; wrong metrics in China; questions about preprints - Retraction Watch at Retraction Watch

  9. joeldshore

    I try to keep in mind the advice of one of my postdoctoral advisors who said the approach you should take to a referee of your paper is to be “aggressively obsequious”!

    Like

    Reply
  10. Pingback: The one kind of review that really gets my goat | Scientist Sees Squirrel

  11. Pingback: Squirrels, serendipity, and the reach of a blog | Scientist Sees Squirrel

  12. Pingback: How to handle a useless review | Scientist Sees Squirrel

  13. Pingback: Why I sign (most of) my reviews | Scientist Sees Squirrel

  14. Pingback: Where do I find the time to blog? | Scientist Sees Squirrel

Comment on this post:

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s