When is a review paper worth writing?

A couple of things have me thinking about review papers lately.  First, I’ve just published one and I’m about to submit another.  Second, over at EcoEvoEvoEco, Andrew Hendry had some fun figuring out how his citation impact would have been improved had he only ever published review papers rather than primary-science ones.

As Andrew points out, writing reviews brings a lot of career benefits.  Among them:

• They tend to be widely read and heavily cited
• They build your reputation as an expert in the subfield you review
• They draw attention to your primary-literature work (presuming your review cites it)
• They support future grant proposals to fill knowledge gaps they identify.

So the case for review-writing as a career move is strong.  But what about the case for review-writing as a contribution to science?  Not all reviews move science forward much.  It’s easy* to identify a topic, collect 20 years’ worth of papers touching on that topic, and summarize them – but the result may not be worth publishing.

The publishability question often comes up because many reviews evolve from the 1^st chapters of theses.  That’s certainly true for some of mine (for example, Heard 1994 from my own thesis, or Ancheta and Heard 2011 from Justin Ancheta’s).  But while every thesis reviews the literature; not every thesis contains a publishable review.  I’m not disparaging theses that don’t; instead, this happens because the function of the review chapter in a thesis is not always the same as the function of a review in the literature.

So when is a review paper worth publishing?  I think there are two criteria: a weak one and a strong one.

The weak criterion is this: that a particular subfield, or more interestingly a particular question, hasn’t been reviewed for a long time.  As an example: my most recently-published review (Pureswaran et al. 2016) is on the subject of spruce budworm population dynamics.  Eastern spruce budworm is the most important forestry pest in central and Atlantic Canada, and it’s also a canonical example of outbreak population dynamics.  It has major outbreaks every 35 years or so (one is in its early stages now), and despite a truly enormous literature, there is no strong consensus about what governs its population cycles.  My coauthors and I realized that there wasn’t anything recent laying out the basics of budworm biology and how this informs the three rival models (well, classes of models) that purport to explain the cycles.  (There was a much more technical review delving into the details of the modeling, but that one was written for a very different audience than we imagined).  We thought it would be useful to publish a review that would introduce people to the conceptual thinking underlying our current understanding of budworm cycles, how that thinking developed over the last 100 years, and how the most recent work is moving it forward.  I think our review meets that goal, but it does so by summarizing current understanding; it doesn’t (I think) lead to any new understanding by itself.

The stronger criterion (as my last sentence hints) is that writing a review is a good idea if you can synthesize literature on a question and, by doing so, come to some important new insight.  This is the criterion the broadest and highest-impact journals use when considering proposals for review papers.  As I’ve progressed in my career, I’ve put more weight on this criterion and become less interested in the weaker one.  As an example, Justin’s review (Ancheta and Heard 2011, mentioned above) considered the impact of insect herbivory on the population dynamics of rare plants.  There’s a conventional wisdom, partly in the literature but mostly so deeply entrenched that it doesn’t even make it there, that insect herbivores don’t have significant impacts on plant population growth** (with a few interesting exceptions, like defoliators of coniferous trees, including spruce budworm).  Our review establishes pretty clearly that, at least for rare plants, this conventional wisdom is wrong; insect herbivores can have major effects on plant vital rates and, in the few instances where it’s studied, on plant population sizes – right up to local extinctions. The studies showing this, though, are scattered, and it takes a review to see the story they collectively tell.  Finding that story wrings a genuinely new answer from the literature.

How does a review meeting the strong criterion come to be?  In my experience, the idea usually grows from a nagging feeling of dissatisfaction with the literature on some topic.  I’ve read what’s out there, and I can see some kind of pattern, or contradiction, or gap, that the writers of individual papers don’t seem to be picking up on.  That was very much the story of Justin’s review; I had read paper after paper showing strong impacts of insect herbivores on host plant populations, but because these papers were scattered in the literature, none ever seemed to point out that this was a generally important thing that contradicted conventional wisdom.  And to be fair, none of them by itself supported such a conclusion.  Only their sum offers strong evidence that we’ve been looking at herbivory wrong (from a population dynamics point of view).  So despite my claim that there are good reasons not to read too much of the literature, some important insights do come from reading a lot of papers and noticing patterns and contradictions and gaps.

I guess I should go read some papers now.

© Stephen Heard (sheard@unb.ca) December 1, 2016

Many of the thoughts in this post I first expressed in an interview I did some time ago with @AurelieLitReview.  Thanks to her for spurring my thoughts on this.

---

*^For some definitions of “easy”.

**^I blame in part Hairston, Smith and Slobodkin’s 1960 “The World is Green” paper – which almost nobody reads, but everyone takes as establishing that plants aren’t under top-down control by herbivores.  A paper Lynne Remer and I wrote exploring stability of herbivore-plant dynamics was repeatedly rejected on the grounds that our models didn’t matter because herbivores didn’t affect plant dynamics – something that was stated, independently by multiple reviewers, completely without evidence. Grrr.

Advertisement