Last week, I wrote about a US court decision that established that legally, spiders are insects (at least in the jurisdiction of the court in question). The case turned on the “ordinary meaning” of the word insect, or roughly, what a reasonable person could think a non-specialist means by it. I was surprised to learn that many dictionaries allow for definitions of insect that include spiders. Could this be true, I wondered? So I took a poll.
Let’s start with the results, and then later we’ll ask if we should have done that.
Are spiders insects, to readers of the post? Unsurprisingly (since they aren’t*), 88% say no. But that leaves 12% who say yes, and that means dictionaries aren’t wrong in how they handle this: with a primary definition that excludes spiders, but a secondary one marked ‘loosely’ (etc) that includes them. Also unsurprisingly, biologists are more restrictive than non-biologists: 15% of non-biologists would include spiders as insects, but only 6% of biologists. (That 6% would surely dwindle close to zero for entomologists and arachnologist.) It’s always a good idea for us to remember that even terms that don’t sound like jargon often have different meanings inside and outside our fields. (There’s no better example than “significant”, in its statistical vs. ordinary senses.)
Now let’s think about polls. What good are they?
Do poll results settle questions? That, of course, depends on the question. If you want to know whether life on Earth has evolved and is evolving under the force of natural selection, or whether face-masks limit the spread of respiratory disease, poll results are completely irrelevant.** But if you want to know what words mean, poll data are exactly what you need. That’s because words mean what people think they mean – no more and no less. So if there was ever a case for a poll, “Are spiders insects” qualifies.
If you’ve been hanging around Scientist Sees Squirrel long enough, you’ll know that I have an annoying habit of constructing parables – posts that seem to be about one thing, but are really about something else, something more general. (It’s a habit I surely picked up from my father, who was a minister who leaned heavily on parable in his sermons.) So here I go again.
Internet polls are toys. They’re generally constructed and interpreted by amateurs (in this case, me). They draw highly – ridiculously – non-random sets of respondents. My spider poll, for example, will oversample people who object to calling spiders insects, simply because the clickbait-y title will draw in that sort of reader. They’re often poorly constructed, with vague, ambiguous, or leading questions. And those things can be sins of omission – because amateurs don’t know how to do better – but they can also happen by commission, when a poll is deliberately constructed to get the answer the poller wants. So internet polls are toys. Now, toys can also be tools: a nail driven in by a toy hammer will still fasten one board to another. It’s just that you have to be very careful: will the hammer actually drive the nail? Is the driven nail in straight, or did it bend? Am I pushing this whole toy-hammer metaphor too far? In the case of my spider poll, I think I can safely go this far: at least some people think the word insect includes spiders. I might even speculate that, given the poll’s likely bias, that “at least some” is likely bigger than the poll suggests. Going much further, though, would definitely be play.
Here’s the thing about amateur polling. It’s one thing to see it in a blog post, where (I hope) nobody will think it’s the gospel truth. It’s quite another to see amateur attempts at polling in scientific scholarship – in grant proposals, journal papers, theses, and the like. Such attempts are common, and they’re often not pretty. If you’re a scientist tempted to use polling for actual research, for God’s sake get a social-science collaborator! There’s a whole discipline of scholarship around the construction, deployment, and interpretation of polls, and you probably don’t know it. Does your own work, or your own field, involve techniques that you’ve put enormous effort into understanding, refining and truthing? Guess what: so does everybody else’s.
More broadly still, that last point doesn’t just apply to polling. Are you a physicist who wants to build a model of Covid-19 epidemiology? An electrical engineer with a solution to climate change? An ecologist who wants to do textual analysis of Shakespeare?*** Get a bloody in-field collaborator, or at least read the field’s experts extensively and deeply. Calls for interdisciplinary scholarship aren’t calls for folks to blithely tromp over their neighbour’s field without bothering to find out what usually goes on there. Breaking down the walls between two silos needs the occupants of both.
So: will I stop doing polls? No; toy polls are fun. But if you catch me interpreting one without acknowledging that it’s a toy, please bust me on it. And if you catch me tromping (scientifically) in my neighbour’s field, please ask me who I’m with.
© Stephen Heard July 1, 2020
*^See what I did there? I asserted my own definition of insect as if it was the truth. If you think that was appropriate, or if you agree with my definition, you should probably read the original post.
**^To settling the questions. They are of course highly relevant to the interesting sociological questions of how knowledge is established, and when, distressingly, it is not.
***^Yes, that last one was a bit painfully self-relevant. Although I haven’t dipped that far into Shakespeare. Or when I have, I’ve pivoted to Zane Gray. Perhaps surprisingly, I’m not the only person to have connected those two great authors.