Image: Tulips, Vera Kratochvil CC-0 released to public domain.
Last week I reviewed a grant proposal for one of the European national granting agencies. It was an interesting piece of work, which – if funded – would gather probably our best dataset so far to test some longstanding questions in my field. It was ambitious, thorough, and well planned. But it didn’t blaze any particularly new path: the techniques were standard, the questions have been in the literature for decades, and every planned analysis has been done before (albeit with smaller and less suitable datasets).
Before I’d even quite noticed, I found that I’d written a sentence in my review saying “There’s nothing original about the proposed research”. But as I looked at that sentence – and as it glared back at me from the screen – I felt like it was judging me more than the applicant. And it should have.
You see, originality in science is highly over-rated. Granting agencies make a big deal about it, often to the extent of asking referees to score it explicitly. Journals make a big deal about it too – some even flat-out refuse to consider “replication” studies that deliberately seek to confirm or refute an earlier finding:
Dear Cognition, if you don’t publish replication studies, I’m not sure you are a scientific journal, let alone a ‘leader’ in publishing rigorous work. https://t.co/k1tOE37cCO HT @ikschneider pic.twitter.com/5xs2XFhaTc
— Daniël Lakens (@lakens) October 31, 2017
But this is wrongheaded. It’s true, of course, that the mission of science is to learn new things about Nature, and doing so requires that we do original work. But it doesn’t require us to do only original work.
In fact, if we all did only original work, science wouldn’t progress well at all. A science consisting only of original work would be a science built paper-thin and, therefore, paper-weak. Our literature isn’t a big pile of facts – by which I mean that the first study to test a hypothesis or use a new technique should never be thought of as settling the question; the first study is something to build on top of, not to build out from. That’s because any single study may be wrong. That isn’t a bad thing about science; it’s just the way it is, when we do science in a complex and variable universe. A single study can be a false positive (or negative), can be a file-drawer-publication-bias artifact, or can be real but be conditioned by too many hidden moderators to generalize*. If we build understanding out from one result, we may start from false assumptions – leading to studies that don’t even make enough sense to be wrong. Real progress in science is made not with one-of-everything originality but with the consilience of many studies that, together, suggest a common answer to an underlying question**. This requires that we do unoriginal science at least much of the time.
Of course, if we need to do unoriginal science, then we need to publish unoriginal science, and we need to fund unoriginal science. And yet I wrote that offending sentence (“There’s nothing original about the proposed research”), because – well, as near as I can figure, because the supposed virtue of originality spends so much time on our lips and in our ears that it’s hard not to let it settle in our brains. I wrote that sentence despite believing it shouldn’t matter.
Everything comes in threes, right? The very next day after I wrote the offending sentence, a friend came to my office, upset that a newly published paper “scooped” the Great New Idea he was working on. That smarts, there’s no denying it – but the mere fact that he used the word “scooped” made clear he’d drunk the originality Koolaid. That’s two. And three? About a paragraph into writing this post, I realized that it isn’t very original either. And on those grounds I nearly quit. Wouldn’t that have been ironic?
© Stephen Heard November 16, 2017
**^Not necessarily many identical studies. The need for exact replications, I think, is greatly exaggerated. Rather, we want slightly different datasets, slightly different systems, slightly different methods, slightly different takes on the same underlying question. That’s where robustness comes from.