Image: “Transparency”, CC BY-SA HonestReporting.com, flickr/freepress
Note: This is a modestly revised version of my original post, which was not written very clearly. (Yes, I’m aware of the irony.) It was easy, reading the original version, to think I was primarily objecting to journals publishing peer reviews. I’m ambivalent about that (and my arguments below apply only weakly to that situation). It should be clearer now that I’m focusing on authors publishing their peer reviews. If you’d like to see how my writing led folks astray, I’ve archived the original version here.
We hear a lot about making science more transparent, more open – and that’s a good thing. That doesn’t mean, though, that every way of making science more transparent should be adopted. It’s like everything else, really: each step we could take will have benefits and costs, and we can’t ignore real impediments. I worry that sometimes we lose sight of this.
One place I suspect we’re losing sight of it is in the movement for authors to publish their (received) peer reviews. (There are also journals that publish peer reviews, such as Nature Communications; I think this is a lot of work with dubious return on investment, but that’s a topic for another day). What I often see is the suggestion that whenever I publish a paper, I should post the full history of its peer reviews on Github or the equivalent. This lets readers see for themselves all that went into the making of the sausage. It’s worth reading a good argument in favour of this, and I’ll point you to Terry McGlynn’s, which I think puts the case as well as it can be put.
I don’t agree, though. Here’s why I won’t be posting my (received) peer reviews:
-
It probably isn’t legal. Peer reviews, like letters and other unpublished works, probably remain under the copyright of the writer*. Of course, the reviewer could agree to licensing terms allowing an author (or a journal) to post the reviews. I’ve never been asked to do that. If I’d signed a review (as I usually do), the author could contact me and ask permission to post it, but that’s never happened to me either. I think my experience is typical, which means that posting peer reviews will usually be a copyright violation.
- I’m not sure what serious problem it solves. Here are some possibilities (feel free to suggest others in the Replies). First, Nature Communications pitches their postings as a way to reassure people that peer review isn’t broken – but those who claim that it is, I’m afraid, will only cherry-pick posted reviews for evidence that seems to support their position. Second, it’s often claimed that the threat of posting reviews will make reviewers write more civilly. I get as angry as the next person when reviewers are rude or make ad hominem attacks, but editors should be redacting these and in my experience relatively few slip through. Third, posted reviews might shine a light on the shoddy if not fraudulent “review” processes at predatory journals, but this won’t be news to anyone who’s been paying attention – and the authors who post reviews won’t be those using, or being used by, the predatory journals. Fourth, perhaps reading the reviewer-author back-and-forth will help readers evaluate a paper’s claims critically. This seems to be a dubious claim given that commentaries and rebuttals seem to have little impact even when they’re published in journals. After all, in journals, at least commentaries have a good chance of being read, which brings me to my next bullet.
- It’s not clear who will read reviews if we post them. Post-publication peer review systems attract vanishingly few commenters, and this suggests that we really aren’t that interested in transcending the conventional view of the published paper as a one-way communication from author to reader. Maybe we’ll be more likely to read previous peer reviews than we are to post new ones (schadenfreude, anyone?), but I’d be surprised. The deluge of published papers is so huge that we’ve long since accepted we can’t read enough of them. Taking on the reading of posted peer reviews and rebuttals (frequently, combined, longer than the actual paper) will only make this situation much, much worse.
-
It will be a disincentive to review, and to review well. This is actually my most important objection. Peer reviewing is an important part of what makes academia work, but it’s voluntary and time-consuming and, as a result, editors often struggle to find willing reviewers. We’ll struggle more if potential reviewers are aware that reviews are commonly posted. Some people will simply decline, having no interest in being part – anonymously or not – of a back-and-forth that continues, publicly, beyond the acceptance or rejection of the paper. Others will stop signing reviews, being willing to reveal themselves to authors, but not to see their name posted for the world at large. I think the option of anonymity is important, but I’d prefer to see as many reviewers as possible decline that option. Still other people will move their more critical comments into the “comments in confidence for the editor”, making peer reviews less about improving manuscripts and more about gatekeeping. Finally, many scientists – like me – will still review, but we’ll see it as a different and more difficult job. When I write a review, I know I’m speaking to an editor and a few authors, and this lets me take some shortcuts in my text (because there are things I know the author knows). I can write colloquially. I can be candid about other papers the author might be citing, or not citing. Writing a review knowing it will be public changes all that**, making the task harder and more time-consuming, and making me likely to do less of it.
So, would posting our peer reviews make science more transparent? Well, yes, it would; but not that much, and with a set of unintended consequences that I’m sure extend beyond the ones I’ve flagged here. Call me unimaginative; call me conservative; call me a stick-in-the-mud – but I just don’t see a case for a dramatic overhaul of a peer-review system that really works quite well, almost all the time.
© Stephen Heard (sheard@unb.ca) November 1, 2016
Thanks to Terry McGlynn for comments on a draft of this post.
*^Copyright normally comes into existence upon creation of a work, not upon its publication or registration – at least in the US since 1976. (Important note: I’m not a copyright lawyer, and aspects of copyright law vary by country. US law seems best resolved and important to a large set of reviewers.) That means unpublished works have the same copyright protections as papers, books, and so on. This is very clearly true for letters, at least for the US and UK; see here for a clear US explainer. The same seems to be true for diaries and other unpublished written works. The extension to peer reviews doesn’t seem like a complete slam dunk, since one might be able to make some argument that a review is derivative of the paper it’s reviewing, or that it’s some kind of work for hire. I’m unaware of legal precedent on the specific peer-review issue, which is why I say publishing your peer reviews “probably” isn’t legal.
**^To be clear: I’m talking here about the kind of voice, and detail, and polish I’d put on a to-be-public review that I don’t normally put on a private review. I’m not endorsing reviewers who abuse the private nature of the process. Like most people, I claim that I won’t write anything in a review that I wouldn’t say to an author’s face. That’s not entirely true, actually – I sometimes don’t sign a review precisely because I wouldn’t be comfortable saying something to an author’s face. But this is because I fear the author’s bad behaviour, not because I’m worried about my own.
Scientist Sees Squirrel 
I’m having trouble understanding why posting a review would be a violation of copyright. Are you arguing that any time someone publicizes written work, that is a violation of copyright? If someone sends me a birthday card, writes a little something in it, and I photograph it and post it on Facebook, is that a violation of copyright? Reviews are completely derivative works and are given to the authors for the authors’ use. I think maybe the issue of (assumed) confidentiality is being violated, but that is a separate issue.
LikeLiked by 1 person
Unless I’ve missed it it’s never been tested in a court of law who owns the copyright of a review, but it would be an interesting piece of case law. I disagree with Miriam’s point that “reviews are completely derivative works” – reviews sometimes include ideas that the author(s) did not originally consider. In the same sense, book reviews are not derivative works, and either the author or the publishers owns the copyright. Wikipedia (assuming it’s accurate) makes this point:
“For copyright protection to attach to a later, allegedly derivative work, it must display some originality of its own. It cannot be a rote, uncreative variation on the earlier, underlying work. The latter work must contain sufficient new expression, over and above that embodied in the earlier work for the latter work to satisfy copyright law’s requirement of originality.”
https://en.wikipedia.org/wiki/Derivative_work#When_does_derivative-work_copyright_apply.3F
Related to this, then, is another reason to not publish reviews. Very, very occasionally I will draft a beautifully constructed paragraph that captures the essence of the field in which a manuscript is located, or the Big Question that the study is trying to address. And I have been known to recycle such paragraphs (with appropriate contextual changes), in my blog or in papers. To do this after a review has been published would constitute self plagiarism.
LikeLiked by 1 person
I think Steve is right. A copyright automatically goes to the author (even if they don’t claim it or file for it), and in the absence of signing the copyright over, something I don’t think I’ve ever been asked to do when agreeing to review, the copyright stays with the author. Reproducing or publishing in full or significant part a copyrighted work is a violation of the copyright. And from the point of view of copyright, which focuses on arrangements of words, not ideas, in no way is a review a derivative work.
LikeLiked by 1 person
Yes, Miriam, my understanding is that if you post my birthday card to you, you are (technically) in violation of copyright. It does sound silly with respect to a birthday card (and if I send you one, feel free to post it!). But it’s fairly well established with respect to longer letters, diaries, and things like that – see the links in the post. I am not, as I say, 100% sure about a review – but not because they are “completely derivative works”. I would definitely argue that if a review is any good, it’s not a derivative work!
LikeLike
A review is to some degree derivative of the original, but not to the extent that I think it’d be easily categorized as a “derivative work”. Unless it was a very bad review.
More importantly, though, just because the author has the copyright does not mean no one else would have the right to use or re-post. It is not “a violation of copyright” every time one person uses another’s work. As Miriam suggests, there is an expectation that the recipient can and will make use of a review. Whether that implicit license would extend to online posting is dependent on the expectations of the person who sent the review – likely most of them would -not- expect that, but expectations can change (or be formally changed with notice.)
Fair use could also potentially cover posting a review, especially if in order for the recipient to respond or critically analyze the review. I am less clear whether Canadian fair dealing would cover such a re-posting.
LikeLike
Thanks, Nancy, for this. It had not occurred to me that ‘fair use’ could cover posting a review – interesting point. It would make sense to me (although of course, again, I’m not a copyright lawyer!) in the sense of posting a rebuttal letter that includes the text of the review being responded to. Great point. (As for expectations – when I write a review, I definitely expect the author will make use of it. I definitely don’t expect – unless otherwise told – that they will post it online!).
LikeLike
Your birthday card analogy is not really relevant; it’s not as if your photograph is being passed off as a birthday card you are now sending to someone else. There are also fair use clauses in many jurisdictions which would undoubtedly apply in that case.
As for derivative works, copyright law allows for derivative works to be considered as a second, separate entity entitled to copyright protection if the modifications etc are substantial and original. In addition, US copyright law contains the following:
which may (or may not) reflect the nature of some reviews, or at least some aspect of a review.
So, whether something is or is not a derivative work is in itself not the issue; the issue, as others have put it is whether the derivative work contains sufficient originality or not and what that is, is something that is not defined in say the US copyright laws but in a range of case law (certainly subject to interpretation.)
A reasonable argument could be made (I am not a lawyer so take my “reasonable argument” with a huge pinch of salt) that even if a review was considered a derivative work under the terms of US copyright law (and the quote above), the original manuscript and a review of the original are substantially different things and therefore, the review is subject to protection under the law.
Unless you are, or have consulted with, a copyright lawyer and are also prepared to defend any claim made against you, it would be prudent to not risk liability and not post something you haven’t authored.
LikeLike
I agree that your final objection is the most important one. I figure that between conference program committees and journal submissions, I’ve written well over 1000 reviews. But I’d stop immediately if I thought future reviews would be made public. The older I get the more I consider my reviews to be private communications where I’m trying to help the authors.
Having said this, there are many cases where authors have completely ignored my review and I’d be happy to have others see the evidence. But I wouldn’t get to pick and choose which of my reviews becomes public. So, I’d solve my problem by not doing reviews any more.
LikeLiked by 1 person
I’m not sure about the disincentive part, it seems just as likely to be an incentive if you can gain recognition for a review, especially for early career researchers. This of course assumes that names are attached to the reviews, which I presume is not true for every “open” review. To me, that point seems to be a wash.
I think the biggest question is, as you stated, what problem does it solve? When reviewers make suggestions that result in less rigor (e.g. selective reporting of results seems to be the most common ‘bad science’ suggestion) will having that review be open actually fix the problem? Maybe, I don’t know. If we’re trying to “open the file drawer” then posting reviews of rejected manuscripts seems to be more important than for accepted ones, and I doubt that would happen.
Finally, the point about low exposure or readership misses some of the points about open science practices. If there is a benefit down the road to making something more available, the benefit won’t necessarily be seen right away with citations or readership. Rather, the benefit will be its mere availability to the next person who comes down the path and requires access to it. Again, I’m not sure if access to the review really solves a pressing problem, but it might.
LikeLike
I believe pretty strongly, people, especially early career people already do gain recognition for doing peer review. They list them on their CV. And I can guarantee you that every tenure committee I’ve sat on and every tenure letter I’ve written looks at that part of the CV pretty closely. Its a good sign of respect from peers (to be asked) and a good sign of whether they’re being a good citizen.
LikeLike
I think “Post-publication peer review systems attract vanishingly few commenters” is missing the point.
It should go without saying that most papers do not inspire PubPeer comments, because most research literature is not even read by more than a small handful of people outside of the authors. The question is how much of the *problematic but widely cited* literature is being commented on using post-publication peer review.
And that number is unquestionably growing, and the movement has made huge progress in having questionable articles retracted and at having researchers who conduct fraud (and other misconduct) disciplined, fired, sued, or prosecuted. As they should be.
The reason for a publisher to make pre-publication peer reviews available is so that if you come across one of these problematic studies–which I assume you are mostly insulated from as an ecologist because you aren’t swimming in a sea of doctored western blots–you can check whether the reviewers did a good job in pointing out these errors but that the editor let the paper in anyway because it was “flashy”. (Which is almost certainly what happened with Nature’s STAP cell controversy, for instance.)
LikeLike
It has become an article of faith that Peer Review is better than anything else. I have come to doubt it as far as anything REALLY novel is concerned. Also I have had a lot of recent experience of people blocking stuff because they wished they had thought of it themselves (“there is nothing new here” they say, without giving ONE example of what is supposed to be out there). I have solved that by posting preprints, and referencing them in the ms. Editors of most journals are now too busy, disinterested or simply not up to job to do their jobs properly. In the worst cases they are malign: http://dbkgroup.org/on-scientific-censorship-and-bitchiness/. It is worse with grants; papers can be submitted elsewhere, grants much less so. Removing anonymity on GRANTS would be welcome.
LikeLiked by 1 person
Excellent post Stephen – I wholeheartedly agree with what you have written. Judging by some of the comments it does, however, seem that we entomologists live in a less competitive and more collegiate world than some of your other readers. That may be a result of the fact that we tend to publish mainly in the lower impact journals?
LikeLike
Thanks, Simon. You know, I’ve seen the “other fields are more competitive than ecology/evolution/entomology” trope in other places too. It might even be true (although it’s always dangerous to compare perspectives on your own field and another; post coming up on that). For instance, some of my molecular bio colleagues worry about being scooped because they disclose methods etc. in a grant proposal. It’s hard to imagine this happening to one of us. I don’t think it’s about “mainly publishing in lower impact journals” though. At least I hope not…
LikeLike
Well for me a high impact journal is one that is over 2, whereas at some UK RI Univertsities you are acively discouraged form publishing in journals with IFs lower than 4. They would rather the data remained unpublished than be associated with a highly reputable first class international journal that happens to be in a ‘niche’ subject. One of the many reasons why I left Imperial College.
LikeLike
Ouch. Only a handful of my recent papers are in journals >4. (For all the flaws in the impact factor, I do think >4 vs. <1 tells you something). I guess I would also have to leave Imperial College 🙂 (Had they hired me; which, when they had a chance, they did not.)
LikeLiked by 1 person
On the copyright of correspondence, yes, the copyright inherently rests with the writer. You can sell the physical object (card, note, letter) but the writer owns the words. See the court cases around literary figures and biography, litigated for instance in Salinger v Random House.
If you reproduce the card, you would have to meet the criteria of fair use (the purpose and character of your use, the nature of the copyrighted work, the amount and substantiality of the portion taken, and the effect of the use upon the potential market)..To publish reviews, there would have to be transfers of copyright or creative commons agreements.
I don’t think it’s been litigated–but my guess is that reviews (in their current state) might be considered correspondence–as apparently guessed via the card analogy. It’s definitely not a derivative work. While an individual on Facebook gets left alone (the internet sometimes seems to be one big copyright violation), a publisher could be a target of litigation and wouldn’t allow it as standard operating procedure.
From a pragmatic point of view, there’s no way we’d get enough people to agree to review if their reviews were published nor would they have the helpful in-depth line-by-line detail we expect. I realize in reading the comments that in a sense I experience open peer review at least at one journal. I see all the reviews. We don’t allow ad hominem reviews and we don’t ask back people who aren’t helpful to authors. So, other than the sheer volume of papers in the system and people’s time, I don’t feel that review is broken.
I also personally feel it’s a bit unfair to the authors. Through revisions and hard work responding to reviews and recommendations from editors, the authors, often early career or from more isolated institutions, reshape their work to be excellent and maybe experience quite a learning curve. I think it’s great that the authors get to present the polished product rather than have to show the long tail of effort it sometimes takes to get there.
Some of the objections in the Comments would be met by using something like Publons–where reviewers post their reviews.
LikeLike
Thanks, Trish, and good point about Publons. What I like about that model (full disclosure: I haven’t used it, and may be wrong) is that the choice to post or not is up to the reviewer, not the author. This indeed removes most of my objections. Whether there is a big upside or not remains to be seen, of course – as earlier commenters have noted, the fact that posted reviews don’t get read a lot YET doesn’t mean they never will be, or that it’s not worth experimenting with the idea. I’m not super motivated to post my reviews on something like Publons, though; personally, I write reviews for the author and the editor, not for the world at large.
LikeLike
Interesting post. I’d like to push back against the idea that no one would necessarily read the reviews. I’ve followed the journal Biology Direct (https://biologydirect.biomedcentral.com/) for some time and have very much appreciated the dialog between the authors and experts in the field. Certainly the quality of reviews there is typically quite high, and gave me something to model my own reviews after as a starting scientist. And my perception is that these reviews tend to add quite significant scientific insight, which the reviewers get at least some credit for. (Note: I also think there are problems with the Biology Direct model, e.g., lack of rejection.)
LikeLike
For once I disagree with a most of what Stephen writes in this post, and other commentators have raised some of the objections or responses I had. Responding to everything here would result in a length comment so I’ll try to avoid that and perhaps do what Jeremy Fox might like to see and that is post a longer response/comment on my own blog.
One point that responders to Stephen do not mention, which I can make here quickly, is that journals that do practice open peer review don’t appear to be struggling to find reviewers.
Some anecdata:
I am on the editorial board of two journals, one of which posts reviews and any subsequent review discussion if a paper is accepted for publication, the other uses a traditional peer review model. I have had no more trouble getting reviewers for papers submitted to the first journal than I have for the latter. Plenty of submissions on F1000 Research have two or three reviews and often back and forth.
At PeerJ, they run an optional process whereby reviewers can choose to reveal their identity and authors can choose to reveal the entire review process. This is as I say optional, but 40% of reviewers have reveal their identity and 80% of authors have agreed to release the full peer review transcript (Source).
I have also reviewed on several occasions for journals in the EGU stable (run by Copernicus), which practices as fully open peer review process; that my reviews would be read by anyone hasn’t stopped me from reviewing for them, nor has it seemingly made it difficult for editors of those journals to find people willing to review under such a model. (I do not remember now if the reviews here are blinded unless a reviewer names themselves as I sign all my reviews regardless.)
LikeLike
Thanks, Gavin. It could be that empirically, I’m just wrong about dissuading reviewers. (It would dissuade me, or at least dissuade me from signing my review, but maybe I’m unusual (cue Cyndi Lauper). Looking forward to your rebuttal post – I’ll link to it here when you do it.
LikeLike
I don’t think you’re wrong about it dissuading reviewers; that reflects the effect this would have on your decision to review and thus you project that opinion on to some subset of the reviewing population with which you identify. If you are in any way typical of a subset of reviewers, which we can probably agree you are[*], I have no trouble accepting that this would dissuade many people from reviewing. My anecdata likely reflect an enthusiasm for (more-)open peer review among a different subset of the reviewing population.
My point is more that, just because you and others of similar persuasion may be discouraged from this form of open peer review, others still will relish the opportunity to further their ideal/mindset and come in to fill gaps.
Quite what would happen if all journals switched to open peer review tomorrow is anyone’s guess, but that is a truly implausible situation.
[*] I won’t say which, largely because I now find myself in the “old[er generation]” or “curmudgeon” (or other similar adjective) camps quite often too 🙂
LikeLiked by 1 person
@Gavin and Stephen,
As you’ve both noted, you can’t say much about whether publishing reviews (and/or removing reviewer anonymity) would on average make it easier or harder to find reviewers just from noting that some people choose to take up these options enthusiastically. It’s a massive self-selection problem.
In a randomized controlled trial by the British Medical Journal (http://www.bmj.com/content/341/bmj.c5729.long), over half of potential reviewers declined to participate in the trial because, if they did, there was a 50% chance that their signed reviews would be posted publicly on the BMJ website. The discussion indicates that this is a “high” rate, which I presume means “appreciably higher than the usual rate at which BMJ reviewers refuse requests to review”.
FWIW, my own anecdotal experience, like the anecdotal experiences of Stephen and Trish, is that a non-trivial number of people would refuse to review if their reviews were going to be published or likely to be published, even without their names on them. Or, they’d decline non-randomly (e.g., only agreeing to review if they knew the author).
I’m happy for individual journals to choose their own policies on this and for authors and reviewers to vote with their feet. Although it’s easy for me to say that given that no journal I regularly submit to or review for has adopted the Nature Communications policy. In most cases, I wouldn’t mind if my reviews were published and it wouldn’t affect whether I’d do the review or how I’d write it. At least, I don’t think I’d mind–it’s all hypothetical for me because I haven’t yet had the chance to review for a journal with Nature Communications’ policy.
An exhaustive review of the (small, oft-flawed) literature on anonymity and openness in peer review is here: http://blogs.plos.org/absolutely-maybe/2015/05/13/weighing-up-anonymity-and-openness-in-publication-peer-review/
LikeLiked by 1 person
Thanks, Jeremy, very helpful addition.
But I can’t resist teasing you ever-so-gently… so here goes. What, no links to the 6 Dynamic Ecology posts where you’ve previously written on “my” topic and, when I go back and read them, they’re better than what I posted? You’re losing your touch 🙂
LikeLiked by 1 person
Your post today is much better than anything I’ve written about this topic. And you’re too kind to past posts of mine. 🙂
If you’re looking for past posts of mine you can easily improve on, my post today is a candidate, judging from the reaction so far…
LikeLike
Your post today was interesting. I could see exactly where the pushback was going to come from (and I’m sure you could too). I don’t think I could improve on it – at least, not on [it + Replies]. And when things are working well, [it + Replies] >> [it]. As is certainly true for THIS post!
LikeLike
Interesting, I hadn’t thought of the copyright issue. I think there are pros & cons of both sides. It does seem pointless publishing reviews, when you think of other written works/creative arts – novelists don’t publish a companion book of all the editor comments & bits that didn’t make the cut! However, having reviewed a number of times for PeerJ (which has been doing this for a few years already) I’ve always chosen the option to allow my review to be published (although the author has the final say on whether reviews do get published with the paper). And I tend to disagree with your last point – knowing my review would potentially be published with my name on it gave me more incentive to write a good review. I can see how such a system could become an issue and I don’t think it would be useful if it became the norm across all journals. But reviews are an indication of how much you know about your field, so as an ECR I guess I thought it would be a career benefit if my peers could see how I review.
LikeLike
Interesting, Manu – it’s clear that my reaction to my reviews being posted is far from universal, and may even be in the minority. Gavin (I think it was) above suggests there may be an age thing going on here… I admit to being, um, not as young as some of my colleagues 🙂
By the way – I agree it would spur me to write a “better” review, in the sense of fuller, more able to talk to not just the author but to other readers. I’m not sure I’d write something more useful to the author, though; and I’d definitely be less likely to write one at all. So the effect on reviewing (at least for me) would be nuanced.
LikeLiked by 1 person
“I agree it would spur me to write a “better” review, in the sense of fuller”
That’s a very pertinent point: reviews take time; good reviews take more time; good reviews written with half an eye on the public audience would take even more time. And time is often the limiting factor for us. It could also mean an increase in the number of my second-least-favourite reviews, the ones where at least half of it is a summary of what the authors did in tedious detail.
For the record, my first-least-favourite reviews are the ones that just say “reject”. It’s only happened once (earlier this year in fact) but I was deeply, deeply annoyed by it, particularly as it was the third, crucial review!
LikeLiked by 1 person
Re: taking more time to write your review if you know it’s going to be published, IIRC that’s another finding of the BMJ study I linked to above.
LikeLiked by 1 person
There’s an interesting parallel here with the discussion of whether reviewers should be allowed to post the text of their reviews on e.g. Publons. I think the establishment opinion is ‘no’, as the fact that author A submitted to journal B is confidential, and remains that way unless the author makes it public. A reviewer can’t share their comments on A’s article for journal B without breaching that confidentiality.
One interesting perspective is given on the COPE forum about this (http://publicationethics.org/forum-discussion-topic-comments-please-4): reviews are a privileged communication between the reviewer, the journal, and the authors. This view has the act of reviewing as akin to ‘work for hire’, and hence covered by contract law. Neither party can make the ‘product’ public without the others’ consent, particularly when there’s a strong expectation of confidentiality.
LikeLiked by 1 person
Interesting, Tim – I need to think more about the Publons model, which I said in a response above I thought got past most of my objections. You’re right, as an author I’d be uncomfortable with a review of my unpublished MS going up publicly. In part that’s because, as you say, my submission to X wasn’t public; but in part it’s because a good review will contain a lot of information from/about the MS, which is not yet public at all. I was thinking they only posted reviews after the corresponding paper was published, and looking at their web site I see now that doesn’t seem to be the case. Hmm…..
LikeLike
That would be troubling, but I just found this in the Publons FAQ:
“Review Permissions
Publons provides verified recognition for peer review without compromising your anonymity or violating journal policies. Your preferences as the reviewer are combined with the publisher’s, journal’s and any authors before displaying each review.
No matter what settings are chosen, Publons never allows information about the manuscript to be released until the manuscript has been published. Typically this means we require a DOI or a URL for the manuscript before reviews can be published or signed.”
LikeLiked by 1 person
Ah, thanks – I didn’t find that. Yes, I agree, that’s much better than I feared!
LikeLike
What they don’t make clear is that if you submit your paper to Nature and reviewer C trashes it, and later get your paper published in PLoS ONE, Publons will allow reviewer C to post their Nature review – even though the fact that the paper was reviewed at Nature remains confidential.
LikeLike
Oh good point – has this happened? I assumed because the review (and any DOI?) is tied to the journal, and verified by the journal, that it couldn’t get attached to a paper published in a different journal. For your scenario to work, wouldn’t the Nature reviewer have to claim they reviewed it for Plos? And I’d hope Publons verification would catch these cases.
LikeLiked by 1 person
Publons actively encouraged the deposition of reviews for rejected papers when they got started, releasing them when the paper was published elsewhere. I think they may have stepped back from this following pressure from publishers.
LikeLike
Semi-related: this very good old post from Tim Vines questioning whether open peer review solves any real problem, and whether it creates problems worse than those it purports to solve:
http://www.molecularecologist.com/2016/05/the-fourth-reviewer-open-review/
LikeLike
I especially like the second point that it doesn’t solve the real issue. I myself am truly frustrated that I’ve had to adhere to an attitude of reading every paper like a reviewer, simply because I can’t trust this process any more. Posting the reviews will make the task of gaining new scientific knowledge even more inefficient. Related to this is that it’s very difficult to publish a self-critical research article that could be very interesting to read. While these may give a much more honest view of what actually happened.
LikeLike
Pingback: Friday links: The Cubs victory tweet prediction scam, and more | Dynamic Ecology
Pingback: Weekend reads: Frustrated scientists; most brutal rejection ever?; public shaming in science - Retraction Watch at Retraction Watch
Publishing reviews is potentially useful for a number of reasons:
– In the best case, three experts will have read the paper carefully. Their considered views can provide extremely helpful orientation about strengths and weaknesses of the paper that the authors may not detail.
– It prevents editorial excess. Nice examples are the STAP stem cell papers that Nature appear to have published without review of the revised versions. Another would be the Voinnet paper that the Plant Cell published despite holding Vicki Vance’s allegations of misconduct (and which years later was retracted hours after she published her review). The glamour journals would see their favouritism towards some authors exposed or curtailed – hardly a bad thing.
– More generally, the quality of the reviews and editorial process can be evaluated. Surely a good defence against predatory journals.
Like many of these initiatives, not all of the reviews will be read by all of the people all of the time. Nevertheless, it’s a shame to see that information wasted when it can sometimes be genuinely important. And it costs very little to publish it.
Finally, who cares about copyright? It’s a technical problem that should be sorted out, but has no bearing on the principles that should decide this issue.
LikeLike
Well, “who cares about copyright” is an odd argument. I agree it’s technical, but if I am in possession of an anonymous review and I want to post it, it’s not clear how that can be sorted out! And it certainly has “bearing on the principles” – one of which is that the creator of a work (the reviewer) should have say over its use. Don’t you think?
LikeLike
What I meant by the “who cares” comment is that an important discussion about the scientific benefits of publishing reviews has been sidetracked by an uninteresting argument about copyright. (Your reply reinforces that point.)
Of course journals should get referee agreement to publish reviews, ideally at the time of initially accepting the review assignment to prevent subsequent bias in which reviews get published. But that problem is quite easily solved and orthogonal to the real scientific questions.
LikeLike
Yes, in the context of journals publishing reviews, you are right – it is relatively easy to ask reviewers for consent. You may lose a lot of them (empirical data are mixed on that), but there is no technical obstacle.
But my post isn’t about that. It’s about whether I as an AUTHOR should post the peer reviews I receive. (See the line directly above the bullet points). Here copyright is not, I think, my most important argument; but it’s not something one can dismiss out of hand as uninteresting. I create content of various sorts. Some of it I license openly, some I retain rights over – but I get to decide. And that means that copyright – and the ethics it reflects – does matter in the context of author-posted reviews!
LikeLike
I did miss the angle about you self-publishing reviews your papers receive. I must have skipped that part after the intro about the Nature Communications policy – sorry about that. I see the copyright angle, although you could always summarise the referees’ arguments if they weren’t addressed already in the article. In general, however, self-publication of others’ reviewing doesn’t seem like the best mechanism. Only conscientious authors will do it and only the anonymous referees and the journal can verify the reviews.
LikeLike
Yes, I think we’re agreed on that! And yes, the intro about Nature Comms confused a bunch of people. I was trying to explain how the issue came to mind, but I probably shouldn’t have!
LikeLike
Boris – I’ve now revised the post. Enough people had the same misunderstanding you did that it was clearly my fault! Thanks.
LikeLike
Pingback: Why I sign (most of) my reviews | Scientist Sees Squirrel
Pingback: Why I don’t sign (most of) my reviews | Dynamic Ecology