Strong opinions, weak effects: What makes a good title?

If you want your scientific papers to be read and cited, you have to give them good titles. Right? This statement seems utterly uncontroversial – after all, the entire function of a title is to inform and attract readers; and the title is the first piece of your paper a prospective reader will see. It’s not uncommon for someone to make a decision to read a paper (or, more likely, not to read it) based on just a few seconds spent skimming a title or a long list of titles. So good titles matter. Right?

It’s not hard to find strong opinions about what makes a title “good”. Actually, I can find a bunch of strong opinions inside my own head. For instance, I simultaneously believe that declarative-sentence titles (those that state a paper’s conclusion in the form of a sentence) are highly effective and are highly distasteful.* More importantly, these strong opinions are widespread outside my head: they’re in writing books, journal editorials, blog posts, websites, you name it. Good titles matter. Right?

There’s been something of a cottage industry, lately, in attempts to show that – and to show how – good titles matter. As a rule, these take advantage of huge paper-citation databases that let us assess correlations between citation rates and easily-measured features of titles. We might use citation data, for example, to test whether citation rates are higher for papers with longer titles or shorter ones; for titles with or without colons; for declarative-sentence titles vs. question titles; and so on. (For two reasons, nearly everyone looks at citation rather than readership. First, it’s trivial to get citation data, but near-impossible to get readership data. Second, it’s a reasonable proposition that papers that are read but never cited aren’t having much impact on the progress of science.**)

I’ve read a few of these papers, but I was driven to assess this literature a bit more systematically while working on revisions to The Scientist’s Guide to Writing (in aid of its eventually-forthcoming second edition). I was more than a little surprised at what I found. Turns out that for almost any feature of titles you can name, the data suggest that (1) that feature may be associated with increased citation, decreased citation, or neither; and that (2) differences in citation rates, when they exist, are pretty subtle.  In other words: strong opinions; weak effects.

I’ll offer you just one example here: title length. It’s widely claimed that titles should be short. What does the literature suggest?

  • A number of papers support the shorter-is-better claim: shorter titles were associated with higher citation rates by Paiva et al. (2012, in PLoS and BMC journals), Letchford et al. (2015, Scopus-wide), Stremersch et al. (2015, in marketing), Elgendi (2019, across MDPI journals), Costello et al. (2019, in Biological Conservation), Ayres and Vars (2019, in law), Gnewuch and Wohlrabe (2017, in economics), Haslam et al. (2008, in social psychology), and Didgah and Thelwell (2013, in biology, biochemistry, chemistry, and (oddly) social sciences). However, effects varied from weak to extremely weak.
  • A number of other papers found the opposite: that longer titles are associated with higher citations rates. This was reported, for example, by Milojevic (2017, for astronomy and economics but not for ecology, mathematics, or robotics), Habibzadeh and Yadollahie (2010, Scopus-wide), and Jacques and Sebire (2010, in medicine). Milojevic and Habibzadeh & Yadollahie reported weak effects, while Jacques and Sebire reported moderate-to-strong ones.
  • As seems predictable based on the first two bullets, another bunch of studies found no association between length and citation rate, including Fumani et al. (2015, in Scientometrics), Falagas et al. (2013, in medicine) Stremersch (2007, in marketing), Rostami et al. (2013, in Addictive Behavior), Subotic and Mukherjee (2014, in psychology), Jamili and Nikzad (2011, in PLoS journals), Fox and Burns (2015, in Functional Ecology), Nair (2015, in management), Alimoradi et al. (2016, in medicine), Stevens et al. (2019, in urban planning), Antoniou et al. (2015, in medicine), and Murphy (2019, in ecology and entomology).
  • One study (Wesel et al. 2014) reported that the association changes in direction across fields.

So: does title length matter? Yes, no, maybe, and it depends, all at once; but almost always, when effects are detectable, they’re weak.

Title length is not unusual. A summary of the literature would look about the same for question titles, colon titles, hyphen titles, and declarative sentence titles. There’s much less data available for funny titles (watch this space!), although what there is suggests weak effects there too. There’s really just one simple feature of titles for which I can point to moderately strong evidence of a moderately strong effect. That’s taxonomic and regional specificity: titles with scientific names of species or names of restricted geographic regions are associated with lower citation rates. (This may have a straightforward explanation, if that specificity is an honest signal of reduced generality of import.)

Does this mean titles don’t matter? Look, it just can’t mean that. Titles have to matter – it’s simply inconceivable that the title of a paper doesn’t influence its readership (and thus its citation impact). If you doubt this, try titling your next paper “Some observations on the ecology of some species” and see if anyone reads it.***

There are two interesting conclusions I think we can draw here. These aren’t conclusions about titles: they’re conclusions about science, and about scientists, and the ways the latter tackle the former.

First, we’re awfully vulnerable to the streetlight effect: there are dozens of papers considering title length not because we think it’s the most important thing about titles, but instead because it’s easy to measure and easy to extract from available databases. The things about titles that actually influence readers may be a little more complex and a little more elusive. This is not a thing that’s true only about titles. In plant ecology, think how many studies have focused on plant height, and how few on rooting depth… and if you’re not a plant ecologist, I’m sure you can think of an example in your own field.

Second, strong opinions about the merits of title features (Short! Question! Declarative Sentence! Colon!) have persisted have persisted through at least a decade of voluminous data that ought to be undercutting those strong opinions. It’s always struck me as peculiar that as scientists, we define ourselves by a way of thinking that confronts ideas with rigorous evidence – but if those ideas are about the way we do and write about science, rather than its content, well, never mind. For other examples of strong opinions with weak evidence (or evidence of weak effects), you might think about font choices, the evils of Excel, open-access publication, and much more.****

The title of this post is fairly short and has both a colon and a question mark. But this post has a declarative sentence title and (rather bizarrely) 14,000 reads in the last year. Headline: Anecdote Trumps Extensive Data!

© Stephen Heard  March 23, 2021

 Image: A few titles from PLoS One, March 20-22 2021.


*^I find them lacking in nuance and they seem to me arrogant and overclaiming. And yet they also seem like the perfect distillation of a paper, the ideal summary of findings. Yes, these positions are largely contradictory. Yes, I hold both of them anyway. Yes, I contradict myself. I am large; I contain multitudes.

**^Taxonomic papers would be one glaring exception to this. People often don’t cite taxonomic papers (the keys they use to identify the species they work with). They should cite them! But they don’t, so these tend to be low-citation-rate science even though they’re absolutely critical to our making any progress at all in ecology and evolution. Go figure.

***^Observables Upon a Monstrous Head would be a little better. Yes, that’s a real title, a 1665 paper by Robert Boyle. Yes, I’ve mentioned this before. Yes, I’m a little obsessed – but who wouldn’t be?

****^I am fully aware that by mentioning these three in particular, I’m inviting impassioned screeds in the Replies. I fully expect that folks simply won’t be able to help themselves. I’d say “go to it, get it out of your system”, but in my experience folks who are extremely exercised about Excel (for example) never “get it out of their system”.

 

Literature Cited

Alimoradi et al. 2016 Annals of Library and Information Studies 63:74-77

Antoniou et al. 2015 Annals of Vascular Surgery 29:286–292

Ayres and Vars 2019 J. Legal Studies 29:427-450

Costello et al. 2019 Biological Conservation 229:A1-A5

Didgah and Thelwell 2013 J. Informetrics 7:861-873

Elgendi 2019 IEEE Access 7:87977-87986

Falagas et al. 2013 PLoS One 8:e49476

Fox and Burns 2015 Ecology and Evolution 5:1970–1980

Fumani et al. 2015 Annals of Library and Information Studies 62:126-132

Gnewuch and Wohlrabe 2017 Scientometrics 110:1573–1578

Habibzadeh and Yadollahie 2010 Croatian Medical J. 51:165-70

Haslam et al. 2008 Scientometrics 76:169–185

Jacques and Sebire 2010 J. Royal Soc. Medecine Short Reports 1:2.

Jamili and Nikzad 2011 Scientometrics 88:653–661

Letchford et al. 2015 R. Soc. Open Sci. 2:150266

Milojevic 2017 Front. Res. Metr. Anal. 2:2

Murphy 2019 Ecological Entomology 44:593–600

Nair 2015 Scientometrics 107:1331–1359

Paiva et al. 2012 Clinics 67:509-513

Rostami et al. 2013 Scientometrics 98:2007–2010

Stevens et al. 2019 J. Planning Education and Research 2019:1–22

Stremersch et al. 2015 Intern. J. of Research in Marketing 32:64-77

Stremersch 2007 J. Marketing 71:171–193

Subotic and Mukherjee 2014 J. Information Science 40:115–124

Wesel et al. 2014 Scientometrics 98:1601–1615

 

14 thoughts on “Strong opinions, weak effects: What makes a good title?

  1. ScientistSeesSquirrel Post author

    Well, this is closely related to the issue of measuring humour in titles (of which, as I say, more another time). You can’t, of course, measure humour (or cuteness), but you can measure a given audience’s perception of either one. So studies of title humour (few that there are) generally use panels of “scorers”. As a teaser: when you do this, you find surprisingly low concordance among scorers about which titles are funny – or even which titles are presumed to be meant to be funny!

    Liked by 1 person

    Reply
  2. Jeremy Fox

    I think your point generalizes. There are lots of topics on which lots of scientists (including me!) have strong opinions, that just don’t matter.

    How faculty job applicants should format their cv’s is one. I have many strong opinions on the “right” way to do various nit-picky details of cv formatting–despite no good reason to believe that any of those details matter. Indeed, my own anecdotal experience suggests that they *don’t* matter.

    I wonder how much the lack of importance of these things is due to the limited range of variation. Sticking with the cv example, the many cv’s I’ve seen when sitting on search committees have varied in their formatting–but not all *that* much. I’d characterize literally every cv I’ve seen as either fine in terms of its formatting, or else slightly annoying in its formatting, but not so annoying as to affect my evaluation of the applicant even a tiny bit. I can certainly imagine weirdly formatted cv’s that *would* cause me to seriously downgrade my assessment of the applicant. But those imaginary cv’s are *so* weirdly formatted–*so* different from the usual–that I doubt I’ll ever see one that weird.

    Liked by 2 people

    Reply
  3. Jeremy Fox

    Are there cases that work the other way? Cases in which few people have strong opinions about the “right” way to do X, even though doing X the right way actually matters? And even though many people don’t do X in the best way?

    Maybe text on posters is an example? I feel like most posters have way too much text (https://dynamicecology.wordpress.com/2016/03/22/the-one-big-mistake-almost-every-scientific-poster-makes/). And I feel like my strong opinion on this must place me in a small minority (because otherwise, why would most posters have way more text than I think they should?). But maybe I’m wrong to think this matters all that much? Maybe I’m wrong to think that, if your poster was better designed, that it would attract appreciably more viewers, and would spark more and better conversations?

    Like

    Reply
    1. ScientistSeesSquirrel Post author

      I TOTALLY agree with you about poster text. But I wonder if people have the strong opinion “posters ought to have lots of text” or if they instead simply don’t think about it. I think those are two different ways to be wrong 🙂

      Liked by 1 person

      Reply
      1. Salix

        I think it’s often a matter of ‘I have so many things to say’, so they try to fit as much as possible on there (Phenomenon, also seen in slides presentations). And since, most scientists don’t come with a background in graphic design, they may think a poster with less text look better but won’t make the connection as to why theirs (full of text) doesn’t look as neat?

        Like, I’ve seen people saying that ‘less is more’ for presentations and then come up with really cluttered slides… I think some courses in infographics could be beneficial honnestly.

        Like

        Reply
  4. Jeremy Fox

    My favorite example of a rare case in which a paper title presumably had a big effect on the paper’s readership: as of a few years ago, the most-viewed Plos One ecology paper ever was this one: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0007595.

    Maybe it’s still the most viewed Plos One ecology paper ever; I haven’t checked recently. It’s currently over 400,000 views, so presumably it’s right up there even if it’s no longer #1…

    Like

    Reply
  5. Chris Mebane

    “…try titling your next paper “Some observations on the ecology of some species” and see if anyone reads it.” For extra security against someone inadvertently reading it, add a place name, such as “Some observations on the ecology of some species at some place.” That’s more secure than the damned WordPress password that’s never right even when I pull it from the password manager.

    Liked by 1 person

    Reply
  6. cinnabarreflections

    Ultimately, the quality of the research would determine whether or not a paper gets cited, regardless of the quality of the title. On the other hand, I believe there is also a “snowball” effect so that once a paper gets cited, it tends to attract more citations. And, as my friend Dan Herms once told me (in jest), the way to get lots of citations is to write a paper that lots of people disagree with (in reference to his citation classic “The dilemma of plants: To grow or defend”.

    Like

    Reply
    1. ScientistSeesSquirrel Post author

      That’s of course true – except that if nobody reads the paper, nobody can assess its quality. So I would certainly argue that if a good title can attract more readers, that will increase citation impact independent of quality! (Or in other words, it’s a multiple regression).

      Liked by 1 person

      Reply
  7. Pingback: Friday links: gut microbiome vs. securities fraud, rethinking graduate admissions essays, and more | Dynamic Ecology

  8. Pingback: The 2nd edition “Scientist’s Guide to Writing” is available for pre-order! | Scientist Sees Squirrel

Comment on this post:

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.