So, last week Meghan Duffy and I put up what amounted to point-counterpoint blog posts. I sign most of my reviews, while Meg doesn’t sign most of hers; but neither of us is quite sure that’s right. As I’d hoped, we got a bunch of good comments in the Replies on each blog. Here are a few things I learned from them:
- More people sided with Meg than with me – that is, most but not all commenters either said they don’t sign, or argued that it’s better not to sign. This surprised me, actually. I expected the opposite, because I hear a lot of arguments in favour of signing reviews (usually citing transparency and accountability, although those aren’t the reasons I sign). Now I’m thinking that perhaps there’s an apparency bias here: people who argue for transparency and accountability in reviewing may just be louder (I don’t mean that pejoratively) than those who prefer more traditional anonymous review. If so, I think that fits a pattern, in which calls to reform something are louder and more public than calls not to fix what isn’t broken.
- I explained that I sign because I invite authors to contact me for fuller discussion, but admitted that this rarely happens. However, several commenters mention engaging in such discussions after signed review – both as reviewers and as authors. This gives me hope that I’m not just tilting at a windmill.
- The two most common reasons given for not signing were (1) concern about retaliation (conscious or unconscious), and (2) concern that the system devolves into quid pro quo if we sign our positive reviews but not our negative ones. I share (1), and when I don’t sign, that’s why. For reasons I can’t entirely articulate, I’m less concerned about (2). I agree that it’s a hypothetical possibility, but for some reason I think scientists are more likely to engage in (unconscious) retaliation than in mutual back-scratching. I’m aware that this is pessimistic, not to say cynical, and I don’t have any data to support it! I will say that in 30 years of publishing, I have yet to receive a soft review from someone whose manuscript I’d previously reviewed. Perhaps this happens more to authors whose careers are less scattershot than mine?
Something I hadn’t thought of, and that makes me sad, is that women may not sign because they suspect that if they do, their comments will be devalued because of their gender. This is a useful reminder for me that, however much I want to believe otherwise, there are still Neanderthals in science*.
- About two years ago, Hilda Bastian wrote a nice review in the PLoS Blog of empirical studies that have tackled benefits and costs of signing reviews (and of double-blinding). Bottom line: it’s complicated, and effects either way are probably modest. (HT Jeremy Fox).
I often learn from people who comment on my posts; so please keep the comments coming!
© Stephen Heard (firstname.lastname@example.org) March 21, 2017
*^On second thought, that comment is really unfair to Neanderthals, who I’m sure were lovely people. Most of them, anyway.