Weird things (some) ecologists believe: “you can’t infer process from pattern”

Warning: this one’s a little bit niche.

When I’m not writing Scientist Sees Squirrel, or books about writing, or books about Latin names, I actually have a day job: I’m an evolutionary ecologist. I teach, and do research, and read the literature (well, sometimes), and I talk with my colleagues about how we do our science and how we might do it better.

I’ve been doing that last bit since my grad school days, 35 years ago, and there’s an assertion that some ecologists love to make that still makes my head spin every time I hear it. You’ve quite likely heard it too: it’s the assertion that “you can’t infer process from pattern”.  It’s such a weird thing to believe that in between times, I think it must just be a caricature, a straw man, hardly worth commenting on. But every time I’ve settled on that, somebody trots the assertion* out again, and darned if they don’t seem to actually believe it. It happened to me again just last week.

I’m fascinated by folks who believe that “you can’t infer process from pattern”, because (of course) we do that all the time. We have to. And the assertion is wrong – and arguably, I think, deeply naïve – on at least two different levels.

The first level is a philosophical one. All empirical science is based on what we can observe, and that’s pattern. Doesn’t matter whether it’s how many galls we see on a willow branch in the woods or the output of a multi-million-dollar particle detector. We observe things. We don’t get to know about process directly, and we never will. Now, I don’t think this is really what my “can’t infer process from pattern” people actually mean by “process” vs. “pattern”, so this isn’t the slam-dunk rebuttal I used to think it was when I offered it over beer. Because most of the time, what they really mean isn’t anything about process and pattern at all; instead, it seems to be something about experiment and observation.

So that’s the second level on which the assertion is wrong: if its makers retreat to a position that you can only infer process from experiment, not from observation, they’re still wrong. The notion that you can only learn about the world through experiments is so deeply naïve that it’s hard to imagine people holding it – which is why it feels like a straw man (oh, how I wish it was!).** Imagine trying to insist on a purely experimental approach to history, epidemiology, astronomy, biogeography, mathematics, biosystematics, plate tectonics, or macroeconomics. Having trouble imagining that? I’m not surprised.

I think there’s a specific historical root to the “you can’t infer process from pattern” canard, one that’s somewhat unique to ecology. There was a research program in the 1970s (roughly), with Jared Diamond as one leader, that attempted to diagnose interspecific competition from the geographic pattern of species occurrence on islands. More specifically, Diamond pointed to “checkerboard” patterns in island archipelagos where, for a pair of similar bird species, a given island would be home to one species, or the other, but rarely both. (The image at the top of this post is one example, for a pair of flycatchers in an archipelago off New Guinea. For some history, albeit written by vocal opponents of the idea, see this paper among many others.) This research proposal was both a failure and a huge success. It was a failure in that it became clear pretty quickly that you can’t, in fact, infer the past action of competition simply from the non-overlapping geographic distribution of two species. It was, at the same time, a huge success in inspiring many lines of research asking how we can infer the action of competition. This work continues today, and it still includes data coming from codistributional patterns (but not alone).

There’s no question that the competition-from-checkerboards stab at understanding ecology got way out over its skis. It’s completely normal, of course, to see a hypothesis raised and then falsified, with new hypotheses following. But there were some outsize personalities involved, some incautious rhetoric, and the convulsions that resulted in community ecology were pretty dramatic. But to take from this the idea that only experiments can teach us anything about the world – that observational approaches are never worth taking – is a breathtaking overreaction.

Of course we can infer process from pattern. We do it all the time. We have to. We just have to do it with appropriate sophistication and caution.

UPDATE: In case you’re thinking this really is a straw man, it took just a little over an hour for someone to drop this pearl of wisdom on Twitter:Sigh.

© Stephen Heard  February 22, 2022

Image: One of Diamond’s bird checkerboards, in the Bismarck Archipelago.


*^In the first draft of this post I used the word “claim” here, but that wasn’t the right word. When someone makes a “claim”, they allow for the possibility that they might be wrong, and for the appropriateness of some kind of counterclaim or the offering of evidence. That is not, in my experience, what the “you can’t infer process from pattern” crowd is doing.

**^Many folks do, of course, take the more reasonable position that experiments are preferred over observational data when possible. But “you should prefer not to infer process from pattern unless for ethical, logistical, or other reasons experiments are not possible” doesn’t have much of a ring to it.

 

31 thoughts on “Weird things (some) ecologists believe: “you can’t infer process from pattern”

  1. Marco Mello

    Nice reflection. Thanks for sharing your thoughts. The example of the “process vs pattern dilemma” is perfect. I totally agree with you and would add: we learn a lot of dogma in our grad schools in ecology, which are later strengthened throughout our careers. We are such easy prey to scientific dogma, because our education as ecologists has a fragile philosophical basis. Discussions in unofficial and official venues are usually based on confusing or wrong notions of theory, hypothesis, prediction, law, rule, axiom, evidence, and paradigm. This dogma spread happens mainly thanks to colleagues, who act like cult leaders and not scientists. Colleagues focused on certainty, and not doubt, push their ideas and create false dilemmas. Think about most classical discussions in ecology: although their leaders always wanted us to believe that a given phenomenon is perfectly explained by either hypothesis A or B, usually the best explanation involves a mix of both and something else.

    Liked by 3 people

    Reply
  2. Ambika Kamath

    Ohh interesting, I like this! I have a related gripe (unsupported by evidence, but it feels true to me?) that I’m curious to hear your thoughts on: in EEB, we do often infer process from pattern, as you describe, and then, because we don’t often make strong predictions in Discussion sections, we lose the opportunity to make new predictions about pattern if the process we’re inferring were indeed what was going on. Instead of in the Discussion, that work of connecting process to next pattern happens in the Introduction section of some new paper, if it happens at all. So it seems to me that for no good reason, we lose the connecting thread of process and pattern as we build on previous research.
    Is there anything to this at all, or am I being silly?

    Liked by 2 people

    Reply
    1. ScientistSeesSquirrel Post author

      Interesting thought! Makes me want to re-read some of my old Discussions.

      In writing about how to write Discussions, I’ve talked about the subsection about ‘future work’. Often, this is a rather vague “we need to know more about X”. A better way to do this is to say “our results suggest this new idea/hypothesis that could be tested by the following experiment/observation”. I think that’s the kind of thing you’re suggesting we don’t do as much as we should?

      Like

      Reply
  3. Eric Pedersen

    I agree with you on the broad point (all we have, fundamentally, is pattern, even in experiments). However, I do think ecology as a field has relied too heavily on trying to match simple patterns and trying to infer process from them (As you mentioned, checkerboard patterns, or now many SDMs). And we frequently take matches between process and a very broad pattern as sufficient evidence of a given process at work; for instance, testing the intermediate disturbance hypothesis by finding a significant quadratic term in a regression. Also see a lot of the debate around niche vs. neutral models centering around which model can fit the specific details of different SADs more accurately. The biggest offender here are the large number of papers that show a low AIC (or high R^2, or high posterior prob, etc.) when including a predictor, and then inferring that the model matches the data generating process.

    I would really like to see ecology move more towards testing proposed ecological processes by looking for multiple lines of evidence and trying to rule out different processes. This can be done both through experiment and observation, as long as we’re looking for patterns that can be used as severe tests (per Deborah Mayo) of different process-based models. I’ve always been impressed by folks working on insect outbreak research for this reason, as they frequently use multiple lines of evidence (including large and small-scale observations, time series, experiments, and models) to rule out different possible processes and to estimate the effect size of different drivers.

    I’d suggest a replacement term: “Matching one pattern in the data is not a severe test of a process”. Not as catchy, but hopefully it emphasizes the right part of the problem.

    Liked by 2 people

    Reply
    1. Eric Pedersen

      I would also follow up: even in its original form, “you can’t infer process from pattern” holds up as a true statement, as long as we take it to mean “we cannot infer processes uniquely from most patterns”.

      If we are willing to be specific about how a process acts, we can make predictions about what patterns a given process might give rise to, and see if those match empirical patterns. The more predictions you can make, and the more precise the predictions, the better. My favourite example of this is Kevin Drum’s write-up on the lines of evidence pointing to a causal effect of lead poisoning on violent crime rates: https://www.motherjones.com/environment/2016/02/lead-exposure-gasoline-crime-increase-children-health/

      Like

      Reply
      1. ScientistSeesSquirrel Post author

        You had me totally nodding along until the claim that “the sun is blue” holds up as a true statement as long as we take it to mean “the sun is yellow” 🙂 I mean, sure, there are true statements that share many of the same words, but they’re not the same statement!

        Otherwise, everything you say is spot on (and buried in the words “sophisticated and cautious” at the end of my post). Thanks!

        Liked by 1 person

        Reply
        1. Eric Pedersen

          My mental model of the lifecycle of scientific criticisms:

          stage 1: germination: Many researchers make a shared mistake (from misunderstanding, missing a subtlety, or sometimes just cutting corners)
          stage 2: development: one or more detailed critiques of the issue come out, breaking down the nuances of how the mistake arose
          stage 3: maturation: either one of the criticizers or a later summarizer makes up a pithy but imprecise summary of the issue as shorthand (“inferring pattern from process”, “pseudoreplication”, “spatial autocorrelation” etc.)
          stage 4: senescence: Reviewer #3’s everywhere use the pithy version to shoot down any paper they don’t like as long as they can find a vague connection to the pithy version of the issue (“you didn’t account for spatial autocorrelation, and this study is pseudoreplicated!”)

          I see the last stage happening a lot in the review process.

          The pithy (and direct) reading of “you can’t infer process from pattern” is that pattern doesn’t tell you anything about process, and I agree that this reading is mostly used as a way of shooting down observational studies that a reviewer doesn’t like. However, I’d argue that the original concept behind the slogan is “patterns are over-determined, so don’t assume that just because a given process *could* produce a pattern that it *did* produce the pattern”.

          Like

          Reply
            1. Eric Pedersen

              I’ve gone down the rabbit hole here, trying to find the earliest mention of “inferring process from pattern”, and I think I’ve found a perfect illustration of this lifecycle at work.

              Based on Google Scholar, the first mention of the phrase is in “Competition and location strategy in branch banking: spatial avoidance or clustering”, which cites Harvey 1966 “Geographical Processes and the Analysis of Point Patterns: Testing Models of Diffusion by Quadrat Sampling” for the concept. But Harvey never uses the phrase. Instead, he quotes Skellam (1953), stating:

              “[T]hough the passage from cause to effect involves no special logical problem, the reverse process does. Unfortunately we cannot with any certainty arrive at an understanding of spatial arrangement of the points from a knowledge of the frequency distribution alone”.

              Harvey then follows that with: “The problem is simply that several different models may give rise to the same probability distribution”.

              I don’t think I could have imagined a more perfect example of the Flanderization of Scientific Critiques (https://en.wikipedia.org/wiki/Flanderization).

              Like

              Reply
      2. Jeremy Fox

        The lead-crime example is actually a super-interesting example to think about in this context. My own admittedly-imperfect understanding of that literature is that lots of independent lines of evidence line up, but not all. And that the recent jump in violent crime rates in the US might end up weakening the evidence for the lead-crime hypothesis, or at least reducing our best estimate of the effect size.

        Like

        Reply
        1. Eric Pedersen

          I agree that it’s still not settled science, but I do find it compelling because of the multiple lines of evidence at work. Also, the lead-crime theory wasn’t meant as a catch-all model (changes in crime rates are caused by changes in lead), as much as a theory for change in crime in a specific period: the increase in crime rates in the 70s to the 1990s, being driven by lead from gasoline (although it also would predict higher crime rates in places that have local increases in childhood lead exposure).

          The recent increases in crime have corresponded with increases in homicides specifically, not violent crime in general. I haven’t heard any good theories yet on the cause in that spike, but I don’t think it really affects the evidence for the lead hypothesis one way or the other.

          Like

          Reply
  4. Jeremy Fox

    “It was a failure in that it became clear pretty quickly that you can’t, in fact, infer the past action of competition simply from the non-overlapping geographic distribution of two species. It was, at the same time, a huge success in inspiring many lines of research asking how we can infer the action of competition.”

    So, being influential compensates for being wrong? I’ll just leave this here: https://dynamicecology.wordpress.com/2016/04/25/being-influential-doesnt-compensate-for-being-wrong-2/ 🙂

    Like

    Reply
    1. ScientistSeesSquirrel Post author

      You really do have an old post for EVERYTHING, don’t you? 🙂 I love it!

      You wrote: “had that incorrect idea never been proposed, the people who did that good science would have done different good science instead, under the influence of other ideas, some of them correct”. And this is hard to argue with, except that it implies that they could and should have known that the idea was wrong, and pursued some other idea instead – and that everyone else also could and should have known the idea is wrong, so ruling it out has no value. That seems like a pretty high bar to argue for. Does it apply to checkerboards? I’d be open to an argument; but I’m not going to be a pushover on it…

      Liked by 1 person

      Reply
      1. Jeremy Fox

        “And this is hard to argue with, except that it implies that they could and should have known that the idea was wrong, and pursued some other idea instead”

        No. It’s both unfortunate, and unavoidable, that scientists will sometimes be influenced by ideas that turn out to be incorrect. Just because it’s unavoidable doesn’t mean it’s not also unfortunate.

        I certainly do think there are cases in ecology in which wrong ideas became influential even though the ideas were widely and publicly recognized as wrong at the time they were proposed, or very shortly after they were proposed. Think for instance of the idea that you can test Hubbell’s neutral model by fitting it, and alternative models, to species-abundance distributions. It should’ve been obvious from the get-go that that particular attempt to infer process from pattern was a non-starter (see, e.g., McGill 2003 Nature).

        Now, should it have been immediately obvious in the late 1970s that inferring interspecific competition from checkerboard distributions was a nonstarter? I’m not sure, though I suspect one could make that case. But whatever; the idea that one can infer interspecific competition from checkerboard distributions *long* outlived the eventual debunkings. Heck, debunkings are *still* being published in leading journals, which suggests to me that that the checkerboard idea is still influential. I take it you want to give wrong ideas credit for at least being influential, on the grounds that they couldn’t possibly have been recognized as wrong at the time they were first proposed. To which, fair enough. But I think you also need to reckon with the fact that the influence of a wrong idea often long outlives repeated public demonstrations that it’s wrong. Many (most?) people whose work is influenced by wrong ideas can’t easily be let off the hook on the grounds that no one could’ve possibly known that the idea was wrong.

        Liked by 1 person

        Reply
      2. Jeremy Fox

        “You really do have an old post for EVERYTHING, don’t you? ”

        My plan is to become the sort of tiresome internet commenter who says in response to everything “I discussed all this back in [long-ago year], can’t believe you didn’t know that.” 🙂

        Liked by 3 people

        Reply
  5. John Pastor

    Here’s how I have always taught how science is most successfully done:

    1. We see a pattern in the world.
    2. We infer hypothetical processes that might account for the pattern.
    3. If possible, do experiments that shut off some of the processes and see if the pattern goes away or at least changes (these experiments can include observations where the process manifestly does not operate strongly).
    4. Test the logical connection and generality of the pattern with the process using mathematical models.

    I have confidence in results where all these approaches overlap; note that this confidence does not necessarily require 95% confidence intervals, but they sometimes help).

    Liked by 2 people

    Reply
  6. Jeremy Fox

    “Imagine trying to insist on a purely experimental approach to history, epidemiology, astronomy, biogeography, mathematics, biosystematics, plate tectonics, or macroeconomics. ”

    Glad you brought that up, but I wish you’d continued that line of thought. Ecologists who like to infer process from pattern love to bring up MacArthur’s old line (reported by Jim Brown) about how Galileo never moved a star. But if you just stop there, all you’ve established is that other people in other fields can infer process from pattern. Not that ecologists can do so. Rather than just stopping with the observation that there are scientists out there who’ve inferred process from pattern, I really wish ecologists would pay more attention to the details of *how* those scientists did it, and then think hard about if/how ecologists could follow their lead.

    For instance, take continental drift. Alfred Wegener’s inference that continents move around is a famous example of an inference based on many different, independent lines of evidence (see Frankel 1987, 2012 for discussion). It’s a great illustration of Erik Peterson’s remarks upthread. So if you want to point to Wegener as a model example of inferring process from pattern, well, tak your own example seriously! Follow Wegener’s lead, and made a point of drawing on many different lines of evidence. Which, to be fair, some ecologists do.

    Or take astronomer Cecilia Payne’s use of spectroscopy way back in 1925 to infer the chemical composition of stars. That’s a very sophisticated piece of reasoning. Payne used quantitative physical theory (the Boltzmann equation and the Saha equation) to correct observed stellar spectrograms for temperature and ionization. And we know that physical theory is reliable because of various lines of experimental evidence here on Earth. So if you want to hold up astronomy as a model for ecology to follow, well, how about citing some examples of ecological reasoning analogous to Cecilia Payne’s? Can anyone cite any ecological examples of using an experimentally-validated quantitative model to subtract out from observational data all (and only) the effects of some confounding process(es) or variable(s), thereby revealing the effects of the process of interest? I can’t think of any such examples, because ecologists mostly lack the needed well-validated quantitative models. Instead, ecologists usually do some sort of permutation of their observational data. That permutation usually obscures the effect of the process of interest, and fails to remove the effects of other processes. (Aside: I actually don’t know of *any* field of science or social science that has relied as much as ecologists have on permutation tests to try, and mostly fail, to infer process from pattern…)

    Or take economics. Empirical economics relies increasingly, and heavily, on what ecologists would call “natural experiments” to infer process from pattern–but using statistical approaches that ecologists rarely use. I’m thinking of instrumental variables, differences-in-differences, etc. It’s not clear to me that ecologists could adopt those statistical approaches, or that those approaches would work that well in ecology (especially given that there are widespread concerns about how well those approaches work in economics…). My understanding is that those approaches rely on quite strong assumptions, some of which are difficult to check, and others of which seem like they’d be hard to satisfy in ecology.

    Liked by 2 people

    Reply
    1. ScientistSeesSquirrel Post author

      See the word “sophisticated” in the last line of the post… 🙂 Ironically, I was going to add “which would be too long and complicated to flesh out in a post”, but you’ve done a pretty nice job in a post-length comment. Which kind of sounds snippy but it’s not meant to be at all – I’m admiring this!

      Like

      Reply
  7. Jeremy Fox

    So, since we’re all agreed that the checkerboards=competition example is an example of how *not* to infer process from pattern, what are some of the *best* ecological examples of inferring process from pattern? Whether by drawing on many different lines of observational evidence, or by fitting different alternative models to the same observational dataset, or whatever.

    A couple of opening bids: Ives et al. 2008 (inferring the causes of irregular population cycles in Icelandic midges), and Kendall et al. 2005 (inferring the causes of population cycles in the pine looper moth). Not sure if either is the last word on those topics beyond any reasonable doubt. But both quite good attempts, I think.

    Looking forward to someone suggesting West et al. 1997 Science (evolutionary optimization of branching circulatory systems as an explanation for quarter power allometric scaling), so that I can grab some popcorn and watch the argument over whether that particular inference of process from pattern was correct. 🙂

    Like

    Reply
  8. jennifer

    The way to infer processes from pattern is also to have understanding of the underlying mechanisms and a minimal mathematical model of these mechanisms implemented with an optimization procedure. “Minimal” means here that you don’t exceed the knowledge you have from other sources/works. For instance, we started to simulate the valve movements of scallops using a very simple biophysical model representing the antagonistic action of the adductor muscle and the hinge ligament. Then after comparison with experimental data series, and some further development, this allows construction of a robust early-warning system that interprets the health status of individuals with respect to environmental drivers (https://www.mdpi.com/2077-1312/9/10/1084/html). It seems important for a new type of applied ecology to emerge that uses and cultivates a strong fundamental knowledge about underlying mechanisms to avoid speculative interpretations of patterns using only statistical analyses. The effort to do this work is considerable, involving a lot of original mathematical development, which too few ecology journals are ready to publish for various reasons. Building on fundamental properties and experimental data in order to interpret patterns is what physics does all the time, why shouldn’t this be done in ecology too and with its own concepts?

    Like

    Reply
  9. Pingback: Maybe it’s time to stop teaching “the scientific method” in 1st year biology | Scientist Sees Squirrel

  10. Pingback: Yes, good writing matters: empirical evidence! | Scientist Sees Squirrel

Comment on this post:

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.