Are two years’ data better than one?

Photo: Two giraffes by Vera Kratochvil, released to public domain, via publicdomainpictures.net. Two giraffes are definitely better than one.

Ecologists are perennially angst-ridden about sample size.  A lot of our work is logistically difficult, involves observations on large spatial or temporal scales, or involves rare species or unique geographic features.  And yet we know that replication is important, and we bend over backwards to achieve it.

Sometimes, I think, too far backward, and this can result in wasted effort.

Which brings me to the oddness of the number two.  I’m sure you’ve read this study, if you haven’t written it yourself: some experiment or some observation is conducted, and then repeated in a second year.  Or perhaps the experiment is replicated at two different study sites, or in two plots at a site, or for two different species. (I’ve done this myself; as just one example, this study is replicated at two different sites, although only in a single year.)  And yet: two seems, in general, like a really dumb choice.

If I run an experiment in two different years (I’ll stick to “years”, but the argument is identical for sites, species, etc.), I might be thinking about years in one of two different ways*.  First, I might be aware of some interesting difference between the two years – for instance, one was a drought year and the other a “normal” year.  Second, I might merely be aware that the world is a variable place, and have some expectation that the outcome of the experiment might vary between any two years.  These ideas about year-to-year differences are treated differently, statistically – but in both cases, it seems like two is a dumb number.

In the first case, “year” is what’s known as a fixed effect.  I’m explicitly interested in the contrast between the drought year and the normal year, and I’d like to estimate the effect of drought on whatever we’re measuring.  But of course my two years give me one drought year and one normal year.  With enough replication within years, I can get a really good estimate of the difference between those years – but I can’t ascribe that difference to drought (or anything else) because there’s no replication at the level of drought vs. normal.  So two is really no better than one – despite being twice as much work and expense.

In the second case, “year” is what’s known as a random effect.  I’m conceptualizing different years as essentially random draws from a larger universe of years that I might have studied, with the idea that there’s variation among years in the outcome of my experiment.  This time, I want to estimate the among-year variance component (wanting to know, perhaps, whether it’s big enough that no single-year study means much).  But estimating variances is hard, and estimates of variance based on two data points are very close to meaningless.  So again, two is really no better than one.

And yet, sample sizes of two (years, plots, sites, species, etc.) are abundant in our literature**.  Why?  Not, I think, because we’re collectively unaware of the futility of two.  I doubt that anything I’ve written so far has surprised you.  My best guess is that despite two’s statistical futility, it nonetheless plays a useful role in an effective publishing strategy.  We all know that reviewers will question “unreplicated” studies, and so we repeat an experiment in two years simply to pre-empt that criticism.  Reviewers and editors tend to be satisfied by two years, not because they really think it improves our inference, but because it (1) brings a study into alignment with common literature practice, and (2) gives them enough cover that they don’t have to think of themselves as approving an “unreplicated” study.  We’re all parties to this unspoken social contract: we agree to pretend that two is enough, and we agree to ignore all the other axes along which the study might be “unreplicated”.  (Did we repeat it with two genotypes?  In two different months in each year?  With two different brands of fertilizer?  Using two different capture or marking techniques?  I could go on; but you’d probably prefer I didn’t.)  We implicitly agree on two because it smooths our publishing interactions***.

Is this cynical?  Yeah.  Am I worried that my next manuscript may be rejected by a reviewer who points out we’ve only done two years, and cites this post?  Absolutely.

© Stephen Heard (sheard@unb.ca) July 25, 2017


*^If I’m “thinking” about years at all, but I guess part of my point is that I’m probably not, really.

**^I was starting to sift through my own papers to see how many times I’ve done this myself.  But before I got far, I realized that I’d rather not know. I know, that makes me a bad person.

***^If you watch The Big Bang Theory, you’ll know that Sheldon can object to some action because it’s objectively and logically silly (gift-giving, for instance).  But if he’s told that it’s a “non-optional social convention”, he simply shrugs and climbs happily on board. Two may simply be a non-optional social convention in scientific publishing.

Well, THAT could have been awkward: job-interview edition

What, Andrew MacDonald asks, do you do if you and a friend are interviewing for the same job?  Academia is a small world, and so this is not a question that can be counted on to stay safely hypothetical.  It has, in fact, happened to me.  Awkward?  Maybe a little.  Especially once you hear the rest of the story. Continue reading

Wonderful Latin Names: Salacca zalacca 

Images: Salacca zalacca, botanical print from unknown source, presumed public domain; via Swallowtail Garden Seeds.  Salak fruits by Midori CC BY-3.0 via wikimedia.org.

 Latin names have a reputation as horribly difficult to pronounce.  Sometimes this is true: I’ve worked on the moth Gnorimoschema gallaesolidaginis for over 20 years, and I still don’t know if I’m pronouncing it correctly.  But other Latin names roll wonderfully off the tongue: the clove tree Syzygium and the hoopoe Upupa epops, for example.

Few roll as wonderfully off the tongue as Salacca zalacca, though. Continue reading

Robert Boyle’s Monstrous Head

Every now and again, a paper is published that’s so peculiar, or so apparently irrelevant to any important question, that it attracts derision rather than citation.  Perhaps it picks up a Golden Fleece Award, or more fun, an IgNobel Prize; or perhaps it just gets roundly mocked on Twitter*.  Much more than every now and then, a paper gets published that just doesn’t seem to connect to anything, and rather than being derided it’s simply ignored.

Perhaps you think this kind of thing is a recent phenomenon.  Continue reading

Canada’s 150th, and how should we think about incomplete progress?

Image: Map of Canada by Pmg via Wikipedia.org, released to public domain.

Canada is 150 years old today, and there will be parties, and speeches, and fireworks.

I’m Canadian, and proud of my country – we’re mostly progressive, mostly supportive of diversity and human rights at home, and mostly a force for good abroad in the world.  We’re also mostly getting better on all those axes.  But we aren’t perfect on any of them, and like everyone, we have darker history (both pre- and post-Confederation) than we’d like.  Continue reading

Thoughts from a room on the 13th floor

Photo: elevator buttons © Shane Adams via flickr.com CC BY 2.0

 Last month I went to my favourite conference (the Canadian Society for Ecology and Evolution), and checked into the conference hotel.  The desk clerk gave me room 1310, and I headed for the elevator and pressed the button for the 13th floor.  And then I did a double-take.  The 13th floor?  I don’t remember ever staying on a 13th floor; in North America, at least, buildings usually hop from the 12th floor to the 14th with only a mysterious lacuna in between.

Nothing untoward happened to me on the 13th floor, of course.  But my stay in room 1310 made me think about the 13-is-bad-luck superstition, and what it says about the human concepts of the universe.  What kind of thinking is behind our usual no-13th-floor convention?  First, we have to believe that the universe is constructed such that the 13th of something is disfavoured.  Second, there has to be some agency (whether natural law or supernatural) omniscient and omnipotent enough to keep track of what things are the 13th of something (floors, days, whatever), and to punish us for being on those things.  And third, that same omniscient and omnipotent agency has to be dumb enough to be hoodwinked by our labelling the 13th of something “14”.* Continue reading

Poll: where do you stand on asking for feedback on unsuccessful job applications?

This post is jointly written by Steve Heard and Jacquelyn Gill, and appears in addition on Jacquelyn’s blog The Contemplative Mammoth

A couple of weeks ago, one of us (Steve) posted “How to write, and read, a (job) rejection letter”. (I should clarify that we’re talking here about the university/college academic job market*).  One piece of advice to job candidates got some interesting pushback on Twitter, including from Jacquelyn.  It was this piece:

Finally, realize that the letter isn’t an invitation to further conversation…Don’t contact the letter writer, or anyone else, to ask for further feedback (not even “so I can improve my future applications”).  Believe me, we understand how much you want that feedback, because when we were in your position we wanted it too.   But the same confidentiality considerations that kept the letter short and a bit vague apply to later conversations too. 

Continue reading