So, last week Meghan Duffy and I put up what amounted to point-counterpoint blog posts. I sign most of my reviews, while Meg doesn’t sign most of hers; but neither of us is quite sure that’s right. As I’d hoped, we got a bunch of good comments in the Replies on each blog. Here are a few things I learned from them:
- More people sided with Meg than with me – that is, most but not all commenters either said they don’t sign, or argued that it’s better not to sign. This surprised me, actually. I expected the opposite, because I hear a lot of arguments in favour of signing reviews (usually citing transparency and accountability, although those aren’t the reasons I sign). Now I’m thinking that perhaps there’s an apparency bias here: people who argue for transparency and accountability in reviewing may just be louder (I don’t mean that pejoratively) than those who prefer more traditional anonymous review. If so, I think that fits a pattern, in which calls to reform something are louder and more public than calls not to fix what isn’t broken.
- I explained that I sign because I invite authors to contact me for fuller discussion, but admitted that this rarely happens. However, several commenters mention engaging in such discussions after signed review – both as reviewers and as authors. This gives me hope that I’m not just tilting at a windmill.
- The two most common reasons given for not signing were (1) concern about retaliation (conscious or unconscious), and (2) concern that the system devolves into quid pro quo if we sign our positive reviews but not our negative ones. I share (1), and when I don’t sign, that’s why. For reasons I can’t entirely articulate, I’m less concerned about (2). I agree that it’s a hypothetical possibility, but for some reason I think scientists are more likely to engage in (unconscious) retaliation than in mutual back-scratching. I’m aware that this is pessimistic, not to say cynical, and I don’t have any data to support it! I will say that in 30 years of publishing, I have yet to receive a soft review from someone whose manuscript I’d previously reviewed. Perhaps this happens more to authors whose careers are less scattershot than mine?
-
Something I hadn’t thought of, and that makes me sad, is that women may not sign because they suspect that if they do, their comments will be devalued because of their gender. This is a useful reminder for me that, however much I want to believe otherwise, there are still Neanderthals in science*.
- About two years ago, Hilda Bastian wrote a nice review in the PLoS Blog of empirical studies that have tackled benefits and costs of signing reviews (and of double-blinding). Bottom line: it’s complicated, and effects either way are probably modest. (HT Jeremy Fox).
I often learn from people who comment on my posts; so please keep the comments coming!
© Stephen Heard (sheard@unb.ca) March 21, 2017
*^On second thought, that comment is really unfair to Neanderthals, who I’m sure were lovely people. Most of them, anyway.
Scientist Sees Squirrel 
Re: more people agreeing with Meg than you: As of this writing, my follow-up poll has 212 respondents, of whom 70% say they never sign their reviews: https://dynamicecology.wordpress.com/2017/03/20/poll-on-contacting-reviewers/
LikeLike
D’oh!
LikeLike
What are your thoughts on the argument from Brian and other commenters (including me) that authors with questions about how to interpret or respond to reviews should be contacting the AE, not the reviewer? Thread starts here: https://dynamicecology.wordpress.com/2017/03/20/poll-on-contacting-reviewers/#comment-57311
LikeLike
This was in the original post, but buried in a footnote where it could be easily missed! I said “It’s been suggested to me that journals might not appreciate my opening the door to author-reviewer discussion outside the formal peer review system. When I have my editor’s hat on, such a thing wouldn’t bother me – but in case it bothers other editors, I make my invitation to the authors explicit, right where I sign my review. I’ve never been asked to take it out.”
LikeLike
Whoops, my bad, should’ve remembered that you’d already addressed that.
LikeLike
Not at all – the very fact I buried it in a footnote indicates I didn’t think it was important that everyone see it. Which I think now I was wrong to do, as it’s an interesting argument.
LikeLike
Arriving late to this interesting post. I have signed reviews ever since I started doing them, but for shifting and slightly overlapping reasons. As a graduate student/young researcher, I signed out of a strong belief I should say whatever I had to say to the author’s face rather than behind a veil of anonymity, but also out of no small desire to take credit for work that I had done. Reviewing can be challenging and, depending on the paper, can result in some pretty interesting thought experiments. I wanted be recognized for those efforts and ideas, especially while I was breaking into field. Nowadays, I sign simply in the spirit of openness.
LikeLike