Archived: original version of “Please don’t make science transparent by publishing your reviews”

NOTE: This is the original version of my Nov 1, 2016 post “Please don’t make your science transparent by publishing your reviews”.  It’s here only so I can be transparent about having made revisions – and perhaps to offer a lesson about how to write unclearly!  (The very first posting also included as screenshot of the Nature Communications announcement – that, of course, really favoured misunderstanding.)

I’m late to this one: for nearly a year now, Nature Communications has been publishing papers and publishing their peer reviews with them*The editorial announcing this trumpeted “Transparent peer review at Nature Communications”, and who could argue with transparency?  Indeed, over the last few years I’ve seen a number of calls for doing this – most not from journals (for which this is a lot of work with dubious return on investment) but from individual advocates of “open science”.  Usually, the suggestion is that whenever I publish a paper, I should post the full history of its peer reviews on Github or the equivalent, so that readers can see for themselves all that went into the making of the sausage.  It’s worth reading a good argument in favour of this, and I’ll point you to Terry McGlynn’s, which I think puts the case as well as it can be put.

I don’t agree, though.  Here’s why I won’t be posting my (received) peer reviews:

  • It probably isn’t legal. Peer reviews, like letters and other unpublished works, probably remain under the copyright of the writer**.  Of course, the reviewer could assign that copyright to the journal, or could specify licensing terms allowing journal or author to post the reviews.  I’ve never been asked to do either (presumably I will be, if I ever review for Nature Communications).  If I’d signed a review (as I usually do), the author could contact me and ask permission to post the review, but that’s never happened to me either.  I think my experience is probably typical, which means posting peer reviews will usually be a copyright violation.

  • I’m not sure what serious problem it solves. Here are some possibilities (feel free to suggest others in the Replies).  First, Nature Communications pitches it as a way to reassure people that peer review isn’t broken – but those who claim that it is, I’m afraid, will only cherry-pick posted reviews for evidence that seems to support their position.  Second, it’s often claimed that the threat of posting reviews will make reviewers write more civilly.  I get as angry as the next person when reviewers are rude or make ad hominem attacks, but editors should be redacting these and in my experience relatively few slip through.  Third, posted reviews might shine a light on the shoddy if not fraudulent “review” processes at predatory journals, but this won’t be news to anyone who’s been paying attention.  Fourth, perhaps reading the reviewer-author back-and-forth will help readers evaluate a paper’s claims critically.  This seems to be a dubious claim given that commentaries and rebuttals seem to have little impact even when they’re published in journals.  After all, in journals, at least commentaries have a good chance of being read, which brings me to my next bullet.
  • It’s not clear who will read reviews if we post them. Post-publication peer review systems attract vanishingly few commenters, and this suggests that we really aren’t that interested in transcending the conventional view of the published paper as a one-way communication from author to reader.  Maybe we’ll be more likely to read previous peer reviews than we are to post new ones (schadenfreude, anyone?), but I’d be surprised.  The deluge of published papers is so huge that we’ve long since accepted we can’t read enough of them.  Taking on the reading of posted peer reviews and rebuttals (frequently, combined, longer than the actual paper) will only make this situation much, much worse.
  • It will be a disincentive to review, and to review well. This is actually my most important objection.  Peer reviewing is an important part of what makes academia work, but it’s voluntary and time-consuming and, as a result, editors often struggle to find willing reviewers.  We’ll struggle more if reviews are commonly posted.  Some people will simply decline, having no interest in being part – anonymously or not – of a back-and-forth that continues, publicly, beyond the acceptance or rejection of the paper.  Others will stop signing reviews, being willing to reveal themselves to authors, but not to see their name posted for the world at large.  I think the option of anonymity is important, but I’d prefer to see as many reviewers as possible decline that option.  Still other people will move their more critical comments into the “comments in confidence for the editor”, making peer reviews less about improving manuscripts and more about gatekeeping.  Finally, many scientists – like me – will still review, but we’ll see it as a different and more difficult job.  When I write a review, I know I’m speaking to an editor and a few authors, and this lets me take some shortcuts in my text (because there are things I know the author knows).  I can write colloquially.  I can be candid about other papers the author might be citing, or not citing.  Writing a review knowing it will be public changes all that***, making the task harder and more time-consuming, and making me likely to do less of it.

So, would posting our peer reviews make science more transparent?  Well, yes, it would; but not that much, and with a set of unintended consequences that I’m sure extend beyond the ones I’ve flagged here.  Call me unimaginative; call me conservative; call me a stick-in-the-mud – but I just don’t see a case for a dramatic overhaul of a peer-review system that really works quite well, almost all the time.

© Stephen Heard ( November 1, 2016

 Thanks to Terry McGlynn for comments on a draft of this post.

*^With an opportunity for the author to decline; and, I certainly hope, with a very clear notice to the reviewers that accepting the invitation to review means consenting to this publication.

**^Copyright normally comes into existence upon creation of a work, not upon its publication or registration – at least in the US since 1976.  (Important note: I’m not a copyright lawyer, and aspects of copyright law vary by country.  US law seems best resolved and important to a large set of reviewers.)  That means unpublished works have the same copyright protections as papers, books, and so on.  This is very clearly true for letters, at least for the US and UK; see here for a clear US explainer. The same seems to be true for diaries and other unpublished written works.  The extension to peer reviews doesn’t seem like a complete slam dunk, since one might be able to make some argument that a review is derivative of the paper it’s reviewing, or that it’s some kind of work for hire.  I’m unaware of legal precedent on the specific peer-review issue, which is why I say publishing your peer reviews “probably” isn’t legal.

***^To be clear: I’m talking here about the kind of voice, and detail, and polish I’d put on a to-be-public review that I don’t normally put on a private review.  I’m not endorsing reviewers who abuse the private nature of the process.  Like most people, I claim that I won’t write anything in a review that I wouldn’t say to an author’s face.  That’s not entirely true, actually – I sometimes don’t sign a review precisely because I wouldn’t be comfortable saying something to an author’s face.  But this is because I fear the author’s bad behaviour, not because I’m worried about my own.