Why most studied populations should decline

Figure: Time series for two populations, each fluctuating in size. At time zero, I start a long-term study, and can choose either of the two populations (open circles). At some other time, I recensus (closed circles).  Red arrows show net population change.

On any given day it’s hard not to notice another headline about a population in decline.  Amphibians are in decline, songbirds are in decline, bumblebees are in decline, fish stocks are in decline.  Nature is under relentless human pressure, both direct and indirect, and before I proceed to make my point today, I need to be very clear that this pressure is real and severe and I don’t doubt for a moment that it’s driving down population sizes of many, many species.

But there’s a very simple but pervasive statistical problem with the data behind population declines.  It doesn’t seem to be widely acknowledged, perhaps because it’s a bit difficult to deal with (if you can point me to some literature contradicting this claim, please do; use the Replies). (UPDATE: Thanks to David Steen for pointing out one clear acknowledgement, in Joe Pechmann et al’.s 1991 Science paper on amphibian declines.)   Put most plainly: when we gather time-series data for populations, we should nearly always expect them to decline.  Not because researcher activity inflicts harm (although that may sometimes be true), but because of a simple – indeed, trivial – interaction between population dynamics and research logistics.

I’ll explain what I mean in cartoon form.  Imagine that I’m going to start a long-term study of some species – let’s call it the lesser purple-snouted crompus.  My study could have many research foci, but let’s assume I at least include estimates of population size.  Crompus populations (like pretty much every population I know about) fluctuate through time; and local populations are imperfectly correlated.  The figure below (or above the post, it’s the same one)  shows the two crompus populations available to me, so when I start my study I can choose to work in a population with plenty of crompi* or one in which crompi are rare.  For many obvious logistical reasons, I’m far more likely to do the former (starting at the hollow circle in the top population) than the latter (the X’d out circle in the bottom population).  So (in my cartoon), I start at what looks like maximum crompus density (as I’d tend to do if I had a choice of many more populations than two), and as a result, almost no matter when I wrap up my study (solid dots), I conclude that there has been net population decline (red arrows).

population time series

I’m not claiming this is a brilliant insight; but it is important because it constitutes a logistical bias** pushing our data toward apparent declines.  And sure, my crompus example is a cartoon; a number of things may weaken the logistical bias in real studies.  I’ll name a few here, but you may think of others.  Longer-term studies may last through many population fluctuations; these will be just as likely to find net declines, but more likely to document the ups and downs to put those declines in context.  Studies that start in large but not maximal populations will sometimes show net increases.  Studies using multiple populations will by definition have to start in more than just the largest one.  And a few long-term studies use locations that are chosen independently of current population sizes (for instance, I think the Christmas Bird Count and the Breeding Bird Survey escape the logistical bias, although the Great Backyard Bird Count does not).  But I think it’s impossible to argue that the bias isn’t real and fairly pervasive, because the only way to avoid it is to sometimes begin studying crompi in a place that there aren’t any – and this is not a feature of a promising grant proposal.

Because the logistical bias seems like a fairly obvious thing, I’ve been surprised over the years that when I ask questions about it, reactions range from bemusement to hostility.  I first asked such questions many years ago, of friends who were studying frogs and were publishing papers on amphibian declines (back when those declines were front-page news).  My friends were busily reporting that the ponds in which they’d started their long-term studies had many fewer frogs than they used to; but when I asked how many frogless ponds they started long-term studies in, they seemed to think I was making fun of them.  I wasn’t. It just seemed clear to me (as it still seems now) that we can’t estimate long-term population trends unless we sometimes study frogs (or crompi) in places that there aren’t any. (UPDATE: I got this reaction despite the clear mention of the logistical bias in the Pechmann et al. Science paper, which I now realize likely predated even my earliest questions, and certainly predated most of them!)

About now I should probably revisit this point: I have no doubt at all that human pressures have left many populations in serious long-term decline.  It’s just that if we want accurate estimates of these declines, we need to account for the logistical bias.  We should want those accurate estimates, and we generally say that we do – but I sometimes wonder, and there’s a more general point here.  My experience is that in scientific discussions of conservation problems, suggestions we might be overestimating a problem are often not very welcome.  A person making such a suggestion might be accused of being a Pollyanna or a corporate shill (I’ve been called both). On the other hand, if we exaggerate a problem a bit, well, many people seem to think that just makes the case for taking action a bit more convincing, and that’s a good thing.  I  think this conflates conservation the science with conservation the action, and in doing so it just inflicts yet more bias on our data.  Many things are amiss in this world; we needn’t paint them as worse than they are – not with the logistical bias, and not with anything else.

All this isn’t in any way an attack on conservation biology. It’s yet another way in which conservation biology as a careful and modern science is a critical foundation for conservation action.  Recognizing and acknowledging the logistical bias is easy enough, which is why I’m surprised it isn’t more widely discussed.  Fully accounting for it will take some work, but it’s nothing a discipline full of smart people can’t do.  Or, I hope, have already done, even if it isn’t obvious to me.

© Stephen Heard (sheard@unb.ca) August 2, 2016

UPDATE: Trevor Branch points out an interesting and important corollary for fisheries management (I’ll paraphrase his tweets here).  If new fisheries tend to start on larger populations (and why wouldn’t they), then there will be a tendency for stocks to be first exploited when they are unusually large (with respect to their long-term population dynamics).  In fisheries, it is (apparently) common to use pre-fishing biomass as a benchmark, and base sustainable catches on it.This may lead to quotas being set inappropriately high!


*^Hey, I made crompi up, I can decide how to pluralize them.

**^Throughout, I’m using the word bias in its statistical sense, not its pejorative one.  A bias is simply something that makes the expected value of an estimate differ in a particular direction from the true underlying value.  In this case, the logistical bias makes our estimate of ΔN more negative.  The operation of this bias doesn’t make us evil – unless, I’d argue, we’re aware of it and deliberately ignore or hide it.

^(Footnote added with update). The Pechmann paper reports long-term population abundance data for four amphibians in a single ephemeral pond (Rainbow Bay) in South Carolina. Over the study period, all four species fluctuated dramatically, and both increases and decreases were observed. In discussing the difficulty of distinguishing natural fluctuations from declines, Pechmann et al. state “Large populations may be more likely to be noticed or used by researchers. Anecdotal data may therefore be biased toward observing peak populations that eventually will decline, rather than the reverse.”  This is a crystal clear statement of the logistical bias, and it’s interesting (in this light) that the paper does not explain why Rainbow Bay was originally chosen as a study site.

Advertisements

25 thoughts on “Why most studied populations should decline

  1. jeffollerton

    It’s an interesting idea, Steve, but my initial reaction is that there is bias in the way you are presenting the evidence for bias! To quote your text:

    “I start at what looks like maximum crompus density (as I’d tend to do if I had a choice of many more populations than two)”

    OK, you’re choosing a population that has the _highest_ population density, but that’s not necessarily the _maximum_ density that can be achieved by that population. So in your top figure the population size never exceeds the initial starting point, but how reasonable is that as an assumption?

    Also, conclusions about the trend within a population can only be made following frequent (at least annual) censuses over a _reasonable_ time period; no good ecologist would sample just twice and then draw conclusions about the population, would they?

    I realise that you refer to these weaknesses yourself, but I think you dismiss them too easily when you say that “most” populations will show biased declines.

    On the other hand I definitely take your point about including sites (ponds or whatever) where the species is absent.

    Like

    Reply
    1. ScientistSeesSquirrel Post author

      Well, I did say it was a cartoon to illustrate! And I covered the “what if I only start in a large, but not maximal, population” objection in the first paragraph below the figure 🙂 The “frequent censuses over reasonable time period” objection is covered there too.

      So, maybe I dismiss these a bit easily, to make my point. But (while I hope I’m wrong) my experience suggests that too many people *don’t even* dismiss the logistical bias I’m talking about because they aren’t even aware of it!

      Thanks for commenting, Jeff, as always!

      Liked by 1 person

      Reply
  2. Peter Apps

    This is one of those insights (hindsights ?) that is so obvious after the fact that palm prints are left on foreheads. The bias may be spread more widely than estimates of population trends – can we confidently extrapolate from abundant study populations to populations in marginal habitats, or with more predators, competitors and diseases when we study behaviour, life histories, physiology, etc ?

    But how would you word a grant application that proposed to study populations where scarcity reduces your opportunity to actually gather any data ?, and if your study animals are scarce how would you ever gather the huge bucketfuls of data required by the currently fashionable general linear modelling ?

    Liked by 1 person

    Reply
    1. ScientistSeesSquirrel Post author

      I’m sure I’m not the only one who has had this insight! (And interesting point about predators/competitors/etc., although I think plant autoecologists at least have always been really interested in marginal habitats)

      Like

      Reply
  3. Margaret Kosmala

    Pretty sure it’s pluralized “lesser purple-snouted crompudes.” From the Greek… 🙂

    But great post. I don’t think many of us stop to think about researcher biases enough.

    Like

    Reply
  4. Victor Venema

    It should be possible to compute how large this effect is on the longer time series you have. Compute the correlation length and do not use the first period with a length of two times the correlation length. Compare these trends to the trends you would get starting at the beginning.

    It makes sense not to start when there is no population. I missed arguments why one would start when the population is very large. Maybe trivial for people working on this field, but not for me as an outsider. Isn’t a population doing poorly also a reason to start studying it? Wouldn’t a young researcher prefer to start with something small (enough) to study with a small budget?

    Like

    Reply
    1. ScientistSeesSquirrel Post author

      Victor – typically, it’s because everything is easier in a population at high density (large extent may or may not make things easier, you have a point there). When crompi are abundant, you spend less time searching for them, you worry less about sampling having an effect on the population, etc. etc. When I collect goldenrod galls, I always prefer looking for them in a place they’re abundant. (And my favourite abundant-gall population “crashed” – which surprised and worried me until I realized it was just the logistical bias in action!)

      A population doing poorly is indeed interesting. But it’s harder and more expensive, not easier and cheaper, to study – counterintuitive though that sounds.

      Liked by 1 person

      Reply
  5. Tom Rooney

    I am glad to see you invoke long-term monitoring projects like the BBS as providing a counter-example. Presumably there are lots of data sets from small mammal trapping projects, botanical surveys, etc. that could be re-purposed to provide a less biased estimate of population dynamics and overcome the lesser purple-snouted crompus problem. People of ecology, hear my plea to make your data available through an online repository!

    Like

    Reply
  6. Manu Saunders

    Very interesting post! I honestly don’t know if I agree or disagree! But you might find this article interesting: http://www.pollinationecology.org/index.php?journal=jpe&page=article&op=view&path%5B%5D=396
    Also, I wonder if it depends on the focal species? Many of the documented ‘decline’ studies have been for threatened or conservation concern species. But in the agriculture literature, there are studies that have documented increases in pest populations, usually in response to changes in land use.

    Liked by 2 people

    Reply
  7. Tobias Jeppsson (@tobjep)

    To me, this is a direct application of the idea of “regression to the mean”, which is a clear problem if not dealt with properly (especially with regard to how you select populations or individuals to study). Or do you think that this is a separate issue? I think there is a paper by Lawton(?) from the 80-90 that deal with this problem in population ecology, but I cannot find it right now. Generally, there are a bunch of papers from different fields that deal with similar problems/applications of regression-to-the-mean. See e.g. Beath & Dobson. 1991. “Regression to the Mean for Nonnormal Populations” (https://www.jstor.org/stable/2337271) and Kelly & Prince. 2005. “Correcting for Regression to the Mean in Behavior and Ecology” (http://www.jstor.org/stable/10.1086/497402).

    The problem can go both ways though – it is probably common to start studying populations both when they are unusually large (pests?) and small (threat of extinction), where the high and low points can just be part of the natural variability. If we then see declines or increases from these levels it might be easy to ascribe this to control measures that has been applied, but in reality it might only be regression-to-the-mean effects that has been observed.

    Liked by 1 person

    Reply
  8. David Skelly

    I agree completely with the author here. Selection bias is a huge problem in the study of population dynamics. Two refs included below from the amphibian world. We were looking at occupancy instead of population density but the same logic applies. In our long term study (Skelly et al. 1999) we saw no net declines in total number of occupancies across 14 species using a design that documented both absences and presences in both survey periods. In a later paper (Skelly et al. 2003), we used our data to estimate how the use of occupied sites as a baseline would bias estimates of species decline. While both papers have been reasonably well cited, the capacity of ecologists to ignore science that disagrees with the dominant narrative has surprised me. As a result, the papers haven’t done much to change the way we estimate or talk about population declines (or increases!) for amphibians or other groups.

    Skelly, D. K., E. E. Werner, and S. A. Cortwright. 1999. Long-term distributional dynamics of a Michigan amphibian assemblage. Ecology 80:2326-2337.
    http://www.jstor.org/stable/176913

    Skelly, D. K., K. L. Yurewicz, E. E. Werner, and R. A. Relyea 2003. Estimating decline and distributional change in amphibians. Conservation Biology 17:744-751.
    http://www.jstor.org/stable/3095232

    Liked by 1 person

    Reply
  9. Mason

    Interesting insights and comments on this important issue. Your amphibian example is excellent, and in hindsight your friends and colleagues studying declines were likely overwhelmed at the time depending on where they were working. The pattern and magnitude of declines observed were much more severe in tropical regions compared to temperate regions. In the amphibian post-decline world we now have ample time to monitor 100s of montane ponds, streams, and forests where few or no amphibians now occur in Central America (where I have experience), fitting one of your premises. In many years of monitoring, few species have recovered and fewer species that disappeared have come back (but there are some good natural recovery/rediscovery stories).

    One statistical way to deal with a scenario where you study a population of zero (that may grow via emigration), or any population that using on count data are zero-inflated models. Just a thought and it would be interesting to go back and analyze old time-series population data with these newer methods to see if the older analyses and trends hold.

    Because of the scale and interest in the amphibian decline crises some very nice papers have been done regarding your idea. Here’s a couple of citations and I can send you PDFs if you’re interested.

    Alford & Richards. 1999. Global amphibian declines: a problem in applied ecology. Ann Rev Ecol Syst 30:133-165. They found most amphibian populations decline more than they increase.

    Green. 2003. The ecology of extinction: population fluctuation and decline in amphibians. Biol Cons 111:331-343.

    Cheers.

    Liked by 2 people

    Reply
  10. Julia Earl

    Very interesting discussion. As (yet another) amphibian ecologist, we’ve been talking about how long term data (i.e., long time series) is really essential due to very large fluctuations in population size thanks to the papers mentioned above. I would be interested to see if there is data that shows whether long-term studies tend to start near the peak abundance and what proportion of them do. I’m also curious about how detection probabilities play into the estimates and whether detection probabilities may change through time, altering our perception of population trends, since many early estimates didn’t necessarily adjust for this.

    Like

    Reply
    1. ScientistSeesSquirrel Post author

      Julia – thanks – and that’s a nice and simple approach to quantifying the effect. If sites are selected this way, then (as you suggest) most time series should start high, not low. Great idea. (Related, I think, to a paper mentioned elsewhere in the Replies about most new fisheries starting at times of unusually high stock size). Thanks for commenting!

      Like

      Reply
  11. Brian McGill

    The LPI (Living Planet Index) is the most famous example of generating headlines over 50% of all animals gone in 50 years. This claim is so patently nonsensical (on the I’ve been alive 50 years and there aren’t 1/2 the birds outside my window or in any of my favorite camping spots basis) that I’ve always wondered how it gets taken so seriously. This is a well-meaning and hard working meta-analysis effort to assess changes in abundance. But the potentials for biased filters on which populations we report seem so obvious and have not been rigorously assessed.

    Aside form the one very important one you mention I would add two more:
    1) Publication bias. Its a lot easier to publish a paper saying a population is declining than it is increasing
    2) You talk about a selective bias across sites within a species, but I suspect the same happens across species (even within a site). How many papers throw out rare species as its too hard to work with – they keep showing up as 1 then 0 then 0 then 1 then …

    It is no coincidence that full community long term monitoring datasets find a very different result since they avoid most of these biases (by sampling everything present in a community you can’t preferentially pick species that are high). The North American Breeding Bird Survey clearly shows very little change in abundance over forty years and hundreds of species (more precisely many sites go down and many go up but they net balance each other out) with a few statistically significant but small differences between subgroups. The same with British bird data. And the PREDICTS group just published a paper showing even in heavily human modified habitats – e.g. croplands – the declines are a lot smaller than the LPI 50% (Newbold et al Science this year).

    As scientists we place our credibility in the policy arena at risk if we are not as rigorous about science leading to alarmist claims as we are about other science.

    Liked by 1 person

    Reply
    1. jeffollerton

      Hi Brian – I agree with you about how the LPI has been interpreted by the media, but in fairness to the authors of the LPI, they do make a point of saying that it’s based on a limited number of species (3669 at last count):

      http://www.livingplanetindex.org/home/index

      So most of the bird species outside your home are probably not included within the LPI. But some of those that are (e.g. House Sparrow and Common Starling in the UK) have certainly had huge population declines that I’ve observed personally and are picked up by the LPI.

      With regard to data for British birds, some groups are doing ok but others are not, particularly birds of farmland. The same is true of farmland butterflies, for which we also have good data.

      Like

      Reply
      1. Brian McGill

        Jeff – the British birds are a good example. There IS a 20% drop over the 30 years measured (Inger et al 2014) , but almost all of that is in the first 5 years in some very abundant species, and most of that just in just your two species of house sparrow and common starling (not using a logarithmic scale weights abundant species very highly in studies of total abundance). (And if you had tried to get conservation groups worried about an impending decline in house sparrows and common starlings 30 years ago you would have gotten laughed at). But none the less there is clearly a story there but not necessarily a generic indication of declining birds. And for the last 25 years it has been a fluctuation of just several percent across all species. And 74 species went up while 70 went down (with more rare species going up). In 2004 Thomas et al estimated the median decline across species was 2% in survey grid squares occupied (Thomas et al 2004) The BBS over a 40 year period doesn’t show that 5 year blip – its within a few percent of flat over four decades. The Britihs farmland declines (including the granivore house sparrow) are not surprising giving farmlands themselves are declining in the UK and may represent a return to something closer to pre-human numbers. There is a real danger of cherrypicking negative results unless one does comprehensive studies (all species) and reports the net across that comprehensive group. I trust these full community, less biased numbers much more than I do the LPI. And they give answers not that different than what my crude outside my window tests give (I never claimed to be able to detect 5% changes outside my window).

        Like

        Reply
        1. jeffollerton

          Hi Brian – thanks for the detailed comments. I think that this is a good example of different data sets showing different patterns. However the one I tend to go back to is the BTO/RSPB’s because it shows trends since 1970:

          https://www.bto.org/science/monitoring/developing-bird-indicators

          As I said, farmland birds are a major concern, though woodland birds are not doing so well either. Looking in more detail at the farmland birds, 12 of the 19 species have shown major declines of c. a third or more; one is approximately stable; and 6 show significant increases:

          https://www.rspb.org.uk/forprofessionals/farming/whyfarming/whyfarming/fbi/

          It’s actually the tree sparrow that’s used in the farmland index, not the house sparrow (which is more associated with urban areas).

          Not sure what you mean by “farmlands themselves are declining in the UK” – 76% of the UK is agricultural land and that’s not changed very much since the 1970s (as far as I’m aware) other than a small amount put into environmental stewardship schemes and and even smaller amount being built upon. So I don’t buy the “pre-human numbers” argument.

          We’re having similar debates about declines in numbers of bees, butterflies and other pollinators – these two posts may interest you if you’ve not seen them:

          https://jeffollerton.wordpress.com/2015/06/03/how-good-is-the-evidence-base-for-pollinator-declines-a-comment-on-the-recent-ghazoul-and-goulson-science-correspondence/

          https://jeffollerton.wordpress.com/2015/11/25/butterflies-and-pesticides-a-new-study-and-a-smoking-gun/

          Best wishes,

          Jeff

          Like

          Reply
  12. Pingback: Links 8/6/16 | Mike the Mad Biologist

  13. zero

    It never occurred to me that population studies would be so narrow.

    I have no experience in the field, but I recall an example from a statistics class using a simplified model of foxes and rabbits. Each population was dependent on the other to determine their change in population in the next year. Populations would tend to follow a cycle (many foxes kill most rabbits, foxes starve and most die, many rabbits thrive with few predators, foxes rebound). Charting the annual population of foxes in isolation would be pointless because their numbers swing wildly from year to year. That gives an impression of an unstable at-risk population when the reality is they are subject to a boom and bust cycle with a partner species and the overall cycle is stable and self-regulating. Human intervention would typically mean killing the foxes; during a fox boom that only advances the feedback part of the cycle and has no long-term effect. During a fox bust, however, the next fox recovery can be severely delayed. (For rabbit-lovers, bear in mind that rabbit population explosions lead to rabbit disease and starvation; foxes kill much faster.) This is also a vulnerable point; if every fox is killed (which is easiest during the bust phase) then the species is no longer local (or is extinct if this occurs throughout all habitats). Another species will step in as a rabbit predator and form a new partnership.

    In a model like this (which does not apply to all relationships), the interesting point isn’t ‘what is the annual trend in fox population?’, the real data is ‘what is the fox population at its lowest point and how does that compare to past cycles?’. Observing foxes in isolation still allows you to answer the interesting question even if you don’t have a model for their population or even know what its partner species is/are, provided you are aware that such relationships must exist. I had assumed that research in this field was aimed at producing useful models that could be used to find interesting intersections or phenomena for further investigation, rather than simply to count the number of a thing every year and naively treat that as a clean signal.

    We expect that human populations generally increase and we seem to think that all other populations should do the same. This is not true. ‘Species in decline!’ makes headlines but it’s not automatically meaningful. If I could say ‘The low point of fox populations has been getting steadily lower each cycle’ that would be meaningful and would help direct further investigation. For example, what are fox strategies for dealing with famine, what are alternate food sources, are there other forces driving fox population down besides simple hunger? In order to preserve a species we have to understand what threatens it and how it adapts / survives / perseveres. Maybe we rescind fox hunting permits during a bust cycle, maybe we make shelters (for foxes or for rabbits) in bad years, maybe we set out food (perhaps with vitamins and vaccines) when conditions are particularly tough. On the other hand, maybe we instead breed rabbit herds and release them when the wild rabbit population declines so as to level-off the bottom of the fox population curve and reduce the risk of a wipeout.

    Like

    Reply
  14. Pingback: Summer flashbacks | Scientist Sees Squirrel

  15. Jennifer

    I’m working on a dissertation at the frontier between history and (marine) ecology and I (almost) completely agree with your post. In fact, the opposite can also be true and then a species gets labelled as “invasive”! Ecological practice has historically been susceptible to the quality of the expertise of an individual observer (since earlier field observations can’t be repeated, how do we confirm the objective truth of the measure?). One example that I’ve been working with is about scientific hubris (“I didn’t see it, it didn’t/doesn’t exist”). This gets transmitted as “truth” and “expertise” in different forums, including scientific literature. It is a significant problem in data recovery and reconstructions of ecological time series that is only partly solved by crossing information from other sources and disciplines. I think we should also look to the digital humanities and models of epistemological processes to get a better handle on how well population trends are being described or reconstructed.

    Liked by 1 person

    Reply

Comment on this post:

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s