Every now and again, you see a critique of a manuscript that brings you up short and makes you go “Huh”.
A student of mine defended her thesis a while ago, and one of her examiners commented on one of her chapters that “the Results section is too short”*. Huh, I said. Huh.
I’m quite used to seeing manuscripts that are too long. Occasionally, I see a manuscript that’s too short. But this complaint was more specific: that the Results section in particular was too short. I’d never heard that one, and I just couldn’t make sense of it. Or at least, not until I realized that it fits in with another phenomenon that I see and hear a lot: the suggestion that nobody should ever, ever do their statistics in Excel.
What’s the connection? It’s this: I propose that when we start out to design an experiment (or an observational dataset), our goal should always be to run our statistics in Excel, and to produce a suspiciously short Results section. This proposition may seem odd, so let me explain. I’ll start with Excel.
Excel does a perfectly fine job of running simple statistics – t-tests, one-way or two-way fixed-factor ANOVAs, regressions, things like that. Its statistical capabilities are limited, though. If you need to run a Bayesian GLMM on a dataset with zero inflation and severely unbalanced sample sizes (just an example), you’re going to need to do it in R, or another software package designed to run sophisticated statistics. But there are two ways you can wind up needing to run a Bayesian GLMM with zero inflation and severely unbalanced sample sizes. One is that the ecological situation you’re dealing with is simply too complex for any other model to work. If so, fair enough. But the other way is that you spent insufficient time thinking through experimental design before you ran the experiment or made your observations. I know this is true, because I’ve been guilty of it myself. At least twice, I’ve set up an experiment and then realized only later that I’d accidentally run a split-plot. (#Headdesk.)
When you can (and again, you can’t always), it’s worth investing some time up front to come up with an experimental design that answers your question with the minimum of statistical complexity**. Similarly, it’s worth putting some thought into what you’re going to measure – and later, of the things you did measure, which things actually help tell the story your paper needs to tell. Reporting more variables, and more complicated ones, in more detail will make your Results section longer – but not necessarily better. Remember, everything that goes in to your manuscript represents a request for reader attention and mental energy. Those are limited resources***.
I wonder if my thesis-reading colleague’s reaction betrayed the same kind of thinking that leads scientists to keep writing convoluted, jargon-ridden sentences and using the passive voice. I wonder if it’s a deep-seated suspicion that if it’s simple and straightforward and easy to understand, it can’t be real science. This is nonsense, of course. Some science is complicated and hard to understand; some is not. And our goal, in experimental design, analysis, and writing, should be this: to make our new insights into the world as easy as possible for others to understand. There’s a time when only I know the new thing I’ve learned about the world. It’s a thrill, and one of the best things about being a scientist; but it is and should be an ephemeral thrill, because our job doesn’t end with discovery.
So: a short Results section, and statistics in Excel. These aren’t signs of scientific weakness; they’re signs of exceptionally good experimental design. Let’s aspire to them.
© Stephen Heard November 6, 2017
*^The comment went on to suggest (although not quite in these words) that if the Results section was so short, then the work probably wasn’t interesting enough to be a paper; or alternatively, that my student hadn’t really understood her own work. Why some people write what they write on student theses, I’ll never understand.
***^Someone who reads this post carelessly will object, probably on Twitter, saying that science is intrinsically hard, and that if a reader isn’t willing to work at understanding a paper they don’t belong in science and that writers shouldn’t have to cater to their laziness. They are right in part: science is hard. People like them make it harder. Be one of the ones who makes it easier instead.