What’s your most undercited paper?

Back in February, I asked “What’s your most overcited paper?. That left an obvious question hanging: what, instead, is your most undercited paper? I’m going to tell you about mine, and I hope you’ll tell me about yours in the Comments. You may be worried that this will be an exercise in which I whine that nobody appreciates my work, but in fact that’s not what I have in mind. Well, not exactly*.

In my “overcited” post, I drew a distinction between a statistically overcited paper and an expectationally overcited one. I’ll draw the same distinction here: a paper is statistically undercited if its citation rate is low relative to other papers I’ve published; it’s expectationally undercited if its citation rate is lower than I’d have expected based on its content.

The simplest way to identify a statistically undercited paper is to regress citation count against years post-publication, and look for large negative residuals**. The figure above shows this regression for my publications, with data from Google Scholar (using Web of Science instead gives a slightly shallower slope but no other important difference). My largest negative residual is for paper A, which is Heard and Semple (1988), The Solidago rigida complex: a multivariate morphometric analysis and chromosome numbers: 27 years old, and cited 14 times (including 7 self-citations). But the low citation rate for this paper doesn’t surprise me a bit. It’s a taxonomic revision, and these are undercited as a class; and it’s a revision of a very small group of plants that occur (but aren’t dominant) in North American prairies and are unimportant otherwise. This was perfectly competent and well-motivated work (I claim), but nothing anybody would expect to become a citation classic!

What about expectational undercitation? That’s more difficult, or at least more subjective, but also much more interesting. Here I nominate Paper B, Heard and Remer (2008), Travel costs, oviposition behaviour, and the dynamics of insect-plant systems***. This paper has had just 6 citations in 7 years (two of them self-citations) – and yet I think it reports some very interesting and quite important results.

In our travel-costs paper, we used simulation models to ask a fairly simple question: could insect herbivores regulate plant populations, imposing density-dependence in mortality or reproduction? We found that, indeed, a couple of simple pieces of biology could lead to density-dependence in attack and thus in plant reproduction. First, if female insects experience costs of travel from one host plant to the next, and those costs scale with distance, then sparser populations experience reduced attack. Second, if females behave adaptively to mitigate those travel costs (by laying more eggs per plant when plants are rare, thus reducing lifetime travel costs but accepting more larval competition), an increasing fraction of plants escape attack as the plant population dwindles. So that’s the answer to our question: yes, insect herbivores have the potential to make plant reproduction density-dependent, and so to regulate populations of their host plants. This should be a big deal: the similar potential for predators to regulate their prey and for parasites and parasitoids to regulate their hosts has been a major theme of population ecology since at least the 1920’s (when the Lotka-Volterra predator-prey models were first derived). Since essentially no studies had asked the question about plants, we thought we were filling a pretty big knowledge gap.

So why has our travel-costs paper elicited a collective shrug from ecologists? I don’t know, but I have a small idea and a big idea about it:

  • The small idea: we published our paper in a journal with limited visibility (Theoretical Ecology, volume 1). You often hear people arguing that in this age of electronic searching, journal identity and impact factor don’t matter anymore, and every paper will stand or fall on its own merits. Well, I don’t think we’re there yet.
  • The big idea: our paper answered a question that we weren’t supposed to ask. For reasons I’ve never understood, a remarkably large fraction of population ecologists seem to be convinced that insect herbivores simply don’t affect the population dynamics of their hosts. Sure, everyone accepts that insects damage individual plants, but this isn’t supposed to make any difference to plant population size. If you’re skeptical that such an idea could be so widely held: our travel-costs paper was rejected from six different journals (!), and insects-just-don’t-affect-plant-population-dynamics was an explicit criticism over and over again. This is really very odd, for four reasons. First, nobody ever seem to cite any published paper, or even any data, as a basis for believing that insects don’t affect plant population dynamics; it’s just something everybody knows (maybe it’s a zombie idea?). Second, if plant population dynamics aren’t affected by insect herbivory, it makes one wonder why farmers spend billions of dollars on insecticides to protect harvests consisting of seeds. Third, some famous biocontrol success stories make it obvious that at least some plants can be strongly controlled by insect herbivory. And fourth, if you look for data that actually quantify population-level impact of insect herbivores on rare plants (for which the question is really important), you don’t find much – but what you do find suggests dramatic impacts.  Our review paper establishing this, by the way, is also undercited.

I think there’s a more general point here. I think every field has questions that we all agree we’re supposed to ask, and questions that we’ve all somehow decided aren’t worth asking. (Perhaps Thomas Kuhn’s scientific revolutions are times when we change our minds about which questions belong in which category – although I’m not arrogant enough to think our travel-costs paper should spark a Kuhnian revolution.) Our question about whether insect herbivores could regulate populations of their hosts just isn’t part of the accepted agenda of population ecology as a field. As a result, there was almost no literature for us to connect our paper to, reviewers didn’t seem to think there was any point applying models to the question, and nobody is discovering our paper because nobody is searching for literature on the topic. It’s hard to change the direction of a field, and it might be harder now than ever with our enormous literature and our emphasis on citation rate as a metric for impact. You can think of this as a bootstraps problem, if you like: a question won’t be recognized as important until there are a bunch of highly cited papers about it, but we won’t write or cite papers about a question until it’s widely recognized as important. This is a good recipe for being stuck.

Now, I realize that this is all verging uncomfortably close to “I’m not crazy, I’m ahead of my time, and nobody recognizes my genius”. So I have to acknowledge the alternative possibilities: that insect herbivory really doesn’t ever matter; or that it does, but our paper just isn’t very good. I don’t believe these alternatives, myself; but if you do, you might not be wrong.

So that’s my most undercited paper. What’s yours, and what do you think made it so? Please tell us about it in the Comments.

© Stephen Heard (sheard@unb.ca) May 26, 2015

*Actually, I don’t think my work as a whole is underappreciated. I haven’t published as many papers as some of my peers, but I’m proud of the impact they’ve had, and some have been cited fairly heavily (if you’re curious, here’s my Google Scholar profile). But sprinkled among my well-cited papers are some poorly-cited ones. There are probably a variety of reasons for this, which is a good topic for a future post.

**I suggested in my “overcited” post that a better analysis would take into account factors like research subdiscipline and the type of paper (review, primary paper, etc.). I don’t think that would change the outcome here.

***Yes, I know, it’s paywalled. In the unlikely event you can’t find a PDF copy on the web, shoot me an e-mail and I’ll send you one.


31 thoughts on “What’s your most undercited paper?

  1. Jeremy Fox

    My own old post veering close to calling myself an unappreciated genius:


    Re: the conventional wisdom that insect herbivores don’t matter for plant population dynamics, yes, that is the conventional wisdom isn’t it? Any possibility it traces back to Nelson Hairston Sr.’s 1989 (?) book on ecological experiments? I vaguely recall (and I may be misremembering) that Hairston reviewed the literature on insect herbivore removal experiments at the time and found that they don’t reduce plant biomass that much.

    Your general point at the end is a great point.

    If you’re interested in writing a guest post for DE on the zombie idea (or maybe just undemonstrated idea?) that insect herbivores don’t affect plant population dynamics, drop me a line…


    1. ScientistSeesSquirrel Post author

      You’re right, Hairston and others over the years (at least going back to the 1960 “World is green” paper) have indeed found that insects often don’t reduce biomass much. That, of course, needn’t mean they don’t affect population dynamics – to ridiculously oversimplify, an insect could eat 100% of the ovules, and that might reduce biomass by 0.000001% but recruitment by 100%! But of course there are lots of herbivore removal experiments that show much larger biomass effects. It’s an odd thing, and some day I should do a very careful lit review on its genesis! Thanks for commenting.


  2. davharris

    I have a paper that explained 95% of the variation ecologists had been studying and which basically concludes, “this subfield is done now; ecologists can go do more interesting things now.” People in that subfield apparently didn’t see the paper or didn’t want to change their research program to do more interesting things.

    (Harris et al. 2011, American Naturalist. https://davharris.github.io/files/Harris%20et%20al%202011.pdf)

    My next paper (in review at Ecology) is more ambitious; it tries to reignite the null model wars and get community ecologists to drop a part of their toolbox that they’ve been carrying around for 35 years. I’m not sure if it’ll blow up or be ignored completely.

    (newest version at https://davharris.github.io/files/interactions/interactions.pdf; should be available from biorxiv very soon)


  3. jeffollerton

    There’s a definite insect herbivory theme emerging here: I think my most expectational under cited paper is a short note that came out of my PhD: Ollerton & Lack (1996) Partial predispersal seed predation in Lotus corniculatus L (Fabaceae). Seed Science Research. The abstract sums up the paper nicely (as it should!):

    “Predispersal seed predation may have implications for plant population dynamics and the evolution of plant traits, but assessing the level of seed predation for an individual plant is not always straightforward. Seeds of Lotus corniculatus (Fabaceae) are often only partially eaten by a weevil seed predator, Apion loti. Samples of these seeds were tested for viability and subsequent seedling vigour. A large proportion of these damaged seeds were viable, and the resulting seedlings almost as vigorous as those from undamaged seeds. The ubiquity of these findings, and their ecological significance, is discussed.”

    I thought that the seed predation literature would jump on in because the implications are that we’re over-estimating the effect of seed predators on plant reproductive output. But it’s had 15 citations, rather less than one a year. Perhaps the journal was too obscure for ecologists. Or the results were really not that interesting. But there’s been a few papers published since then basically saying (or rediscovering) the same thing, but not citing my 1996 paper. It’s a topic that probably deserves a mini-review to bring it out to wider attention; I’m not active in seed predation research any longer but I suspect that those who are don’t routinely test the assumption that damaged seeds are not viable.

    Liked by 1 person

  4. Rob Beiko

    Definitely our super-fun paper about Aquifex aeolicus and lateral gene transfer: http://www.ncbi.nlm.nih.gov/pubmed/24281050. There is an ongoing discussion about where the “early branchers” belong, and we found some interesting stories buried in a whole heap of gene trees.

    It doesn’t help that this one of the top three in my “most excruciating papers to get out the door” list 🙂

    Liked by 1 person

  5. Pingback: Friday links: dating tips from scholarly papers, old school scientific fraud, and more | Dynamic Ecology

  6. Paul Caplat (@PaulCaplat)

    Interestingly, my most expectationally undercited paper was also published in volume 1 of Theoretical Ecology (http://link.springer.com/article/10.1007/s12080-008-0021-5). It’s not that I think it deserves great recognition but it had elements of novelty that I really thought would lead to more citations. It took off a bit recently, but I have seen many papers on exactly the same topic that ignored it. When I contacted the authors, they had missed it. So, for citation purpose, a new journal, although of excellent quality, might not be the best place to publish exciting results…

    Liked by 1 person

  7. jeffollerton

    That raises another interesting issue, Paul: what does someone do when authors have clearly missed (your) relevant work? In the past I’ve certainly sent friendly emails to authors, politely suggesting that they might wish to look at some attached PDFs. But I can imagine that not everyone would feel comfortable doing that. It relates nicely to a recent post of mine, though I could also imagine developing that theme as a separate post: https://jeffollerton.wordpress.com/2015/01/08/what-do-academics-do-once-the-research-is-published/


    1. Paul Caplat (@PaulCaplat)

      Hi Jeff, good point. I must say that the paper I refer to was the exact thing that prompted me to contact a high-profile scientist who had missed it in a very similar publication. I was extremely hesitant to do so (being a young postdoc at the time), but his answer was very positive and I am much less hesitant today.
      I find interesting how academic self-promotion is considered in different cultures. All my Australian colleagues do it, none does here in Sweden. But I guess it will only grow as people recognise that busy researchers are grateful to be told directly about an interesting paper when they don’t have time to browse every TOC 😉

      Liked by 1 person

      1. jeffollerton

        Hi Jeremy – it is a tricky issue, agreed. But the question in your poll that directly relates to this suggests a majority support, even though you see it as controversial:

        “Sending a pdf of your work to other people in your field: I expected this one to be controversial, and it was: 63% approve as not self-promotion, 24% had reservations, 13% disapprove as self-promotion.”


        1. Jeremy Fox

          Yeah, in general I’m a little uncomfortable with this practice, which places me in a minority. But really, it depends how it’s done, and how well. If, say, I wrote an ms in which I missed someone’s very important and directly relevant paper on the topic, I’d appreciate them emailing me a pdf with a little note. In the rare cases when I’ve emailed a pdf to a stranger, it’s been in these circumstances (I hope!).

          I think this is what a lot of “self promotion” comes down to–it all depends exactly how you do it. Meg has a good comment on this in the thread on that old DE post.

          Liked by 1 person

  8. Pingback: Are “side projects” self-indulgent? | Scientist Sees Squirrel

  9. Pingback: Creativity, play, and science | Scientist Sees Squirrel

  10. Pingback: What correct scientific idea hasn’t yet proven fruitful or influential, but will in future? | Dynamic Ecology

  11. Pingback: We praise originality, but we don’t seem to value it | Scientist Sees Squirrel

  12. Pingback: Waifs and strays – those papers nobody cites (or reads?) | Don't Forget the Roundabouts

  13. Pingback: My most important work is not what I’ve always thought | Scientist Sees Squirrel

  14. Pingback: Happy 4th birthday, Scientist Sees Squirrel! | Scientist Sees Squirrel

  15. Pingback: It’s been a while since I’ve been this proud of a paper | Scientist Sees Squirrel

  16. Pingback: What should you do when Reviewer #2 says “Cite my papers”? | Scientist Sees Squirrel

  17. Pingback: Moby Dick and scientific writing | Scientist Sees Squirrel

  18. Pingback: Opinion, evidence, and preprints | Scientist Sees Squirrel

  19. Pingback: 2020 was weird for blogging, too | Scientist Sees Squirrel

  20. Pingback: On creativity and science – and play! | Scientist Sees Squirrel

Comment on this post:

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.