A few months ago, I wrote a post that prompted a brief twitter discussion with Meghan Duffy about whether we sign our reviews. I tend to sign mine, and Meg tends not to, but neither of us felt completely sure that our approach was the right one. So, we decided that it would be fun to write parallel posts about our views on signing (or not signing) reviews. Here is Meg’s, over at Dynamic Ecology; please read it, as she makes excellent points (all of which I agree with) even while arriving at a different conclusion (and a different default practice) than I do!
A lot has been written about the merits of signed vs. anonymous peer review. There are arguments on both sides (which I don’t intend to review comprehensively), but in general I’m firmly convinced that at least the offer of anonymity is important to getting broad reviewer participation and high-quality reviews. But I sign almost all of the reviews I write. This seems odd in at least two ways. First, here I am plugging anonymity, but I don’t use it much; and second, if I sign almost all of my reviews, why don’t I sign all of them? I’ll try to explain; and I’m trying to explain to myself as much as I am to you, because I’m far from convinced that I’m doing the right thing.
My reason for signing is not a belief that the review process should be open and transparent in general – that a culture of signing reviews make them “better”, either more constructive or less likely to contain ad hominem attacks or other reviewer misbehaviour. You hear a lot of arguments to that effect, but I’m not convinced by them*. I’m unaware of strong data suggesting that signing improves reviews, or if does, that it’s the only way to achieve that. I’ll grant you, of course, that anonymous reviews can sometimes be nasty. We’ve all gotten infuriating reviews (and some of us have done stupid things as a result), but in my experience as an editor, truly inappropriate reviews are quite rare**. (It’s actually not awful if some inappropriate reviews get written, as long as editors do their jobs and redact them. That some editors don’t do this drives me up the wall.)
My reason for signing is also not that a more localized belief that my own reviews will be better if they’re open (independently of the science-wide argument in the previous paragraph). I honestly don’t think that signing changes my behaviour, in the sense that it ensures that my reviews are constructive and polite. I’m tempted to claim that “I won’t write anything in a review that I wouldn’t say to an author’s face”, but if I’m honest, that isn’t quite true – which is why I carefully specified that I sign most of my reviews. Occasionally, I remain anonymous precisely because I wouldn’t be comfortable saying something to an author’s face. But: this is not because I’m worried that in saying it, I’m behaving badly. Instead, it’s because I’m worried about an author behaving badly in response – whether active retaliation or just an unconscious bias against me when my own work is up for review. I think the former is very rare; but the latter probably isn’t. We all remember when our work is criticized; and it’s only human to let those memories sway your future judgement even if you’re intellectually resolved that it shouldn’t***. My occasional anonymity lets me be as critical as I think is needed. I still try to phrase that criticism as constructively as I can, of course, and most reviewers do the same.
This watch-out-for-retaliation logic suggests a likely effect of experience on review-signing strategy. I’m tenured and thus impervious to the most serious kinds of damage; early-career researchers have a different cost-vs.-benefit situation and might wisely remain anonymous more often than I do. I’d worry that initiatives to require signed review would reduce reviewer diversity by discouraging early-career researchers (who make terrific reviewers) – although the admirable commitment of many ECR to the open-science movement might suggest this is less of a problem than I think.
So if it isn’t to improve science in general, and isn’t to improve my own reviews, why do I sign? It’s actually quite simple: so that authors who want to can contact me for fuller discussion of my review****. Maybe what I’ve written is unclear. Maybe the authors would like to try out a proposed fix on me before committing themselves to a full manuscript revision. Maybe what I’ve written suggests something to the authors that neither they nor I thought of, and that they’d like to chat about. If an exchange of emails can get a paper revised better, or in one step rather than a repeated series of submissions and new reviews, then it seems like everyone wins.
Peer review has two functions, and they’re quite different. The first is gatekeeping – providing some meaning to the credential supplied when something appears in the peer-reviewed literature. I don’t need to sign to provide that function. The second function, and the one I find I’m more interested in, is manuscript improvement. Peer reviewers always make my science better – and as a reviewer, my ambition is always to do the same for another author. Signing my reviews is one way I try to make my reviewing more effective, because it means the review itself can be the beginning rather than the end of a conversation about how a manuscript could be better.
Here’s the thing, though – I’m not sure it works. In practice, few authors ever contact me. I haven’t kept track, but it has to be less than 1 in 20. If so, then there seems to be little benefit to my signing. That leaves only the costs. I’ve figured this out, but I still (almost always) sign. I guess I have a choice: I can think of myself as foolish, for sticking with a practice that cost-benefit analysis can’t support; or I can think of myself as selflessly tilting at a worthwhile windmill. You can probably guess which way I lean.
© Stephen Heard (firstname.lastname@example.org) March 15, 2017
UPDATE: Here’s an older but very relevant post by Tom Webb, coming down on Meg’s side. (And with a nice bonus Friends reference.)
*^There are even good arguments that even if signing helps reviewers write reviews better, this is offset by the fact that it leads authors to read and respond to them worse. As Hope Jahren put it on Twitter: “As an Editor I always liked it best when reviewers did not sign. Once somebody’s name was real, it shifted focus from the content of the review and the MS itself [to] personalities: ‘well, s/he *would* say that, because s/he is/isn’t XYZ ..’ For a bad anonymous review authors would reply with ‘reviewer believes we said X and we really said Y’. For a bad identified review authors would reply with ‘Dr. So&So has always been opposed to the idea of Y’. The former represented a much more productive process for the ultimate growth of the paper, I found.”
**^It’s an apparency thing. The few nasty reviews that do exist get tweeted about, ranted about, shared with friends, and spread widely. The huge majority of helpful ones get left on desktops. When was the last time someone said to you “Hey, let me tell you about the really sensible and helpful thing a reviewer wrote about my paper”?
***^I’m disagreeing here, I think, with Jeremy Fox over at Dynamic Ecology, who thinks we needn’t worry about “disagreeing publicly with Dr. Famous”. I’d like very much to agree with him, but can’t quite.
****^It’s been suggested to me that journals might not appreciate my opening the door to author-reviewer discussion outside the formal peer review system. When I have my editor’s hat on, such a thing wouldn’t bother me – but in case it bothers other editors, I make my invitation to the authors explicit, right where I sign my review. I’ve never been asked to take it out.