When is a review paper worth writing?

A couple of things have me thinking about review papers lately.  First, I’ve just published one and I’m about to submit another.  Second, over at EcoEvoEvoEco, Andrew Hendry had some fun figuring out how his citation impact would have been improved had he only ever published review papers rather than primary-science ones.

As Andrew points out, writing reviews brings a lot of career benefits.  Among them:

  • They tend to be widely read and heavily cited
  • They build your reputation as an expert in the subfield you review
  • They draw attention to your primary-literature work (presuming your review cites it)
  • They support future grant proposals to fill knowledge gaps they identify.

So the case for review-writing as a career move is strong.  But what about the case for review-writing as a contribution to science?  Not all reviews move science forward much.  It’s easy* to identify a topic, collect 20 years’ worth of papers touching on that topic, and summarize them – but the result may not be worth publishing.

The publishability question often comes up because many reviews evolve from the 1st chapters of theses.  That’s certainly true for some of mine (for example, Heard 1994 from my own thesis, or Ancheta and Heard 2011 from Justin Ancheta’s).  But while every thesis reviews the literature; not every thesis contains a publishable review.  I’m not disparaging theses that don’t; instead, this happens because the function of the review chapter in a thesis is not always the same as the function of a review in the literature.

So when is a review paper worth publishing?  I think there are two criteria: a weak one and a strong one.

The weak criterion is this: that a particular subfield, or more interestingly a particular question, hasn’t been reviewed for a long time.  As an example: my most recently-published review (Pureswaran et al. 2016) is on the subject of spruce budworm population dynamics.  Eastern spruce budworm is the most important forestry pest in central and Atlantic Canada, and it’s also a canonical example of outbreak population dynamics.  It has major outbreaks every 35 years or so (one is in its early stages now), and despite a truly enormous literature, there is no strong consensus about what governs its population cycles.  My coauthors and I realized that there wasn’t anything recent laying out the basics of budworm biology and how this informs the three rival models (well, classes of models) that purport to explain the cycles.  (There was a much more technical review delving into the details of the modeling, but that one was written for a very different audience than we imagined).  We thought it would be useful to publish a review that would introduce people to the conceptual thinking underlying our current understanding of budworm cycles, how that thinking developed over the last 100 years, and how the most recent work is moving it forward.  I think our review meets that goal, but it does so by summarizing current understanding; it doesn’t (I think) lead to any new understanding by itself.

The stronger criterion (as my last sentence hints) is that writing a review is a good idea if you can synthesize literature on a question and, by doing so, come to some important new insight.  This is the criterion the broadest and highest-impact journals use when considering proposals for review papers.  As I’ve progressed in my career, I’ve put more weight on this criterion and become less interested in the weaker one.  As an example, Justin’s review (Ancheta and Heard 2011, mentioned above) considered the impact of insect herbivory on the population dynamics of rare plants.  There’s a conventional wisdom, partly in the literature but mostly so deeply entrenched that it doesn’t even make it there, that insect herbivores don’t have significant impacts on plant population growth** (with a few interesting exceptions, like defoliators of coniferous trees, including spruce budworm).  Our review establishes pretty clearly that, at least for rare plants, this conventional wisdom is wrong; insect herbivores can have major effects on plant vital rates and, in the few instances where it’s studied, on plant population sizes – right up to local extinctions. The studies showing this, though, are scattered, and it takes a review to see the story they collectively tell.  Finding that story wrings a genuinely new answer from the literature.

How does a review meeting the strong criterion come to be?  In my experience, the idea usually grows from a nagging feeling of dissatisfaction with the literature on some topic.  I’ve read what’s out there, and I can see some kind of pattern, or contradiction, or gap, that the writers of individual papers don’t seem to be picking up on.  That was very much the story of Justin’s review; I had read paper after paper showing strong impacts of insect herbivores on host plant populations, but because these papers were scattered in the literature, none ever seemed to point out that this was a generally important thing that contradicted conventional wisdom.  And to be fair, none of them by itself supported such a conclusion.  Only their sum offers strong evidence that we’ve been looking at herbivory wrong (from a population dynamics point of view).  So despite my claim that there are good reasons not to read too much of the literature, some important insights do come from reading a lot of papers and noticing patterns and contradictions and gaps.

I guess I should go read some papers now.

© Stephen Heard (sheard@unb.ca) December 1, 2016

Many of the thoughts in this post I first expressed in an interview I did some time ago with @AurelieLitReview.  Thanks to her for spurring my thoughts on this.

*^For some definitions of “easy”.

**^I blame in part Hairston, Smith and Slobodkin’s 1960 “The World is Green” paper – which almost nobody reads, but everyone takes as establishing that plants aren’t under top-down control by herbivores.  A paper Lynne Remer and I wrote exploring stability of herbivore-plant dynamics was repeatedly rejected on the grounds that our models didn’t matter because herbivores didn’t affect plant dynamics – something that was stated, independently by multiple reviewers, completely without evidence. Grrr.


6 thoughts on “When is a review paper worth writing?

  1. Pingback: Friday links: Freeman Dyson vs. community ecology, “white trash” in academia, and more | Dynamic Ecology

  2. Pete

    Interesting post, thanks!

    What are your thoughts on writing a primary research paper paired with a subsequent comprehensive review vs. a single larger paper that includes both primary data and an extensive discussion that mimics a comprehensive literature review but is more condensed?

    I’ve got a smattering of old data that is not in and of itself likely a complete enough “story” for publication in a decent journal, but should be very impactful when when put in context of hundreds of as yet unconnected reports a literature. I think it’s valuable to review those 15 years of papers, since a significant theme is obscured by continued use of non-standard nomenclature. However, since I’ve moved on in my research, it’s unlikely that two papers will ever get done, I am leaning towards a single paper with a mega-discussion, but that seems a tall order to get published.

    Any ideas?


    1. ScientistSeesSquirrel Post author

      Of course it’s hard to have strong opinions without knowing more specifics, but I like the argument that the data may mean a lot more in the perspective of the review. Of course, the key will be to control the size of the combined MS. Very large papers aren’t just harder to get published; I think they’re harder to get *read*, too. So ruthless editing will probably matter! Fortunately, I think the virtues of “comprehensiveness” in a review are less than people often think (so ruthless editing may be less painful than expected). Good luck!



Comment on this post:

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.