I made some raisin buns the other day, and I swear there’s a connection to science coming.
The recipe called for, among other things, 2 eggs, 3½ cups of flour, ½ cup of brown sugar, and 2¼ tsp of yeast. Two and a quarter teaspoons – that’s quite precise, isn’t it? One can imagine a test kitchen industriously experimenting, through dozens and dozens of batches, to nail down just the right quantity of yeast for this recipe. 2 tsp isn’t quite enough; 2½ is definitely too much. But if you bake a lot, you might smell a (metaphorical) rat. If 2¼ tsp is just precisely right for this recipe, why does nearly every yeast-leavened recipe call for the same 2¼ tsp? How can 2¼ tsp be the precisely right amount for raisin buns, and for dinner rolls, and for molasses bread, and for pizza dough, and for – well, you get the picture.
In reality, the precision of 2¼ tsp doesn’t matter. It’s just that yeast is conventionally sold in little foil packets, and one packet contains ¼ ounce of yeast – or 2¼ tsp. Frequent bakers don’t buy the packets, because yeast is much cheaper and more convenient in bulk; but a lot of us still carefully measure out 2¼ tsp, over and over again, for every different recipe, without stopping to actually think about it.
This unthinking conformity to ritual is the very thing you wouldn’t expect to find in science. Scientists prize logical and independent thinking, and have an empiricist’s disdain for the argument from authority. Right? Don’t we?
Maybe not. There’s a strong argument to be made that science is rife with unthinking custom along the same lines as the 2¼ tsp of yeast. There are things we do not because we’ve thought them through and decided that they’re just right, but because they’re what everyone does. Let me give you two examples: one quite field-specific, and one rather general.
For my field-specific example, I offer the use of “leaf packs” to measure decomposition rates in stream ecology. A leaf pack is a bundle of dead leaves, zip-tied to an anchor and usually enclosed in screening, that you place in a stream; after some set amount of time you return and measure how much leaf mass has disappeared. This simple technique has become ubiquitous among stream ecologists (I’ve used it myself), and on the plus side, it’s produced an enormous volume of nicely intercomparable data on decomposition rates. On the minus side: leaf-pack data are often of questionable relevance. Real leaves in real streams don’t decay in stationary packs isolated from the mechanical action of tumbling rocks and other substrate, for one thing. Stream ecologists (including me, when I was one) don’t do leaf-pack experiments because they’re precisely the right experiment for the question they’re asking. Stream ecologists, mostly, do leaf-pack experiments because leaf-pack experiments are what stream ecologists do.*
Leaf-pack experiments are a rather narrowly focused example (and I hope you’ll suggest others, from your own fields, in the Replies). How about something that nearly all of us are guilty of? How about acronyms? Scientific writing is notorious for its heavy use of acronyms. Are they just exactly what’s needed, or are the 2¼ tsp of yeast? The case for acronyms is that they make our writing more compact. Thing is, most of the time, it’s a weak case. Measured by number of characters, acronyms often make our writing only a little bit more compact. Measured the more important way, by effort needed for a reader to digest the prose, they often make our writing less compact. So brevity may be why we tell each other we use acronyms, but I don’t think it really is. The real reason? I’m pretty sure it’s simple: as scientific writers, we use acronyms because that’s what scientific writers do. This starts early, when we “teach” scientific writing by telling our students to read papers from the literature, and to write like them. So that’s what our students do – producing exactly the kind of turgid, colourless, acronym-packed prose that came before them; and that can in its turn be modeled by the next generation to come. This tyranny of circular expectation is, I think, responsible not just for the plague of acronyms but for a lot more of what’s wrong with our literature. If we thought about it carefully, we’d use many fewer acronyms; but we generally don’t think about it. Prose dripping with acronyms just sounds like science to us, so we keep producing it.
Raisin buns, leaf packs, and acronyms. Similar underpinnings – but only the raisin buns are delicious.
© Stephen Heard October 13, 2021
Image: The raisin buns, my photo CC BY 4.0; yeast packets © kiliweb via openfoodfacts.org CC BY-SA 3.0
*^I didn’t choose this example to heap particular shame on stream ecologists – I hope that’s clear from my admission that, while I was doing stream ecology, I did leaf-pack experiments too. Every field has its blind spots. “Wait”, you might be saying, “that’s not a blind spot, it’s a standard method”. Well, it can be both.
Scientist Sees Squirrel 
That’s a very good point, and I wholeheartedly agree with you. We forget that science is a human culture. So, as in all human cultures, big or small, institutional or informal, global or local, conformity comes before critical thinking. We are not special in that matter because we are scientists. Especially when we enter the “scientist’s journey”, we need to put a huge effort into belonging. Therefore, after years and years of training and fighting for a stable position, we incorporate many “2¼ tsp of yeast” in our every day practices without even noticing. Scientists are human after all.
LikeLiked by 2 people
Just gonna drop the “p < 0.05" comment here and run away. 🙂
LikeLiked by 4 people
Great example! It’s possible to adopt a p<0.05 rule through careful thought – but I'm pretty strongly convinced that almost nobody adopts as a result of such thought. See https://scientistseessquirrel.wordpress.com/2015/11/16/is-nearly-significant-ridiculous/, of course!
LikeLiked by 1 person
I use this post of yours when teaching statistics!
LikeLiked by 1 person
As editor of The Scientific Naturalist series in Ecology, I have been STRONGLY discouraging acronyms. Most authors make one up (I suspect they first did this for the field notes, which is fine but not in a paper) and then maybe use them once. Or else, they clog the paper with numerous repetitions of many of them. My little contribution to a more literary scientific writing.
LikeLiked by 1 person
Strong endorsement, John! Thank you.
LikeLike
I’ve seen some of that. I was discussing what tool to use to do an analysis and mentioned papers that were comparing the tools and how well they worked. My colleague used a specific tool because that’s what everyone else uses and the paper has more citations than anything else. I don’t think that’s a particularly strong argument for its use.
I’ve also encountered the acronyms. When I was involved in a paper that was discussing relative humidity, I went through and changed every single “RH” or “%RH” to “relative humidity.” It might be a bit longer but it’s easier to read and helps make the paper just a little bit more accessible.
LikeLiked by 1 person
Huh. My mom–whose dad was a baker–taught me that “cooking is an art, baking is a science”. I think she heard that line from her dad, but I don’t know for sure. But I’d always thought that that line was the conventional wisdom among bakers–that you *do* need to measure everything carefully. Because if you deviate from the recipe more than a little bit, the recipe will come out very differently and probably badly. In contrast, if you’re cooking, you can usually vary the proportions and make substitutions a fair bit, and you’ll usually end up with something that either doesn’t taste too different than the recipe, or else tastes different but still good. Have I been misled my whole life?
LikeLiked by 1 person
I was taught that line too. It seems to be true for some things, like pie crust, but not for many others, like cakes. Hypothesis: you can tell from the recipe when it’s true or not. For pie crusts, recipes are precise in surprising ways: like 3.75 c flour with 6.5 Tbsp of butter and 3.5 Tbsp cold water (I totally made those numbers up, do NOT use them to make pie crust!). But for other things, you find quantities like 2 c flour + 1 tsp baking soda + 1 tsp salt + 1 tsp baking powder. What are the odds, if small deviations were critical, that the perfect quantities would always turn out to involve integer combinations of teaspoons and cups?
So I’ve become somewhat more of a rebel. But my wife still makes the pie crusts, because mine are terrible.
LikeLike
I think it’s important to separate two issues here: the desirability of using methods that work (for some appropriate value of “work”), and the desirability of defaults–of methods that everybody uses, often without having to think about it too much. I take it your worry is that lots of scientists too often default to using a method that doesn’t work? Or is your worry that scientists too often use default methods, independent of whether the default methods work or not?
LikeLiked by 1 person
If I understand your distinction, I guess the latter – that once a default has become the default, we then use it without thinking – including in situation where it doesn’t “work”, or where something else would work better.
LikeLike
Thinking about it further, I find I’m not entirely clear on the distinction myself! So let me think out loud a bit…
One could say that it’s bad to use bad methods, regardless of why they were chosen. If you use a bad method because it’s the default everyone else uses, that’s bad. But if you use a bad method after putting lots of thought into your choice, that’s just as bad.
One could say that the issue is risk. That it’s risky to use a default method, because the method *might* be bad. On average, in the long run, one will choose bad methods less often if one puts thought into one’s choice of methods.
One could say that the issue is one of discovery. If we all default to using established methods, how will we ever discover new ones that might be better?
One could say that the issue is that the widespread use of defaults reduces methodological diversity, thereby magnifying downside risks. It’s unlikely that lots of people would *all* choose a bad method, unless that bad method were the default that everyone just uses without thinking. And if everyone who works on topic X chooses a bad method, well, that’s really bad–much worse than if just a few investigators use a bad method. Of course, one also has to consider that widespread use of defaults also magnifies *upside* by reducing methodological diversity. If everyone thinks for themselves, some people (hopefully not many, but some) will choose some bad method. A default in favor of a good method means that good method will be chosen more often than if it wasn’t the default. So you’d have to weigh upside vs. downside here. Offhand, I’d probably worry that magnification of downside risk would outweigh magnification of upside benefit, but it’s hard to say.
One could argue that there’s some statistical value to everyone using the same method, independent of whether the method is bad or not. Meta-analysis is a lot easier to do, and should report less among-study heterogeneity in effect size, if everyone uses the same methods. And one could of course argue that some ecologists overrate the statistical value of everyone using the same method, independent of whether the method is good.
Finally, you could worry about costs and benefits associated with the time and effort needed to choose a method, regardless of the choice made. Time and effort not spent thinking about which method to use is time and effort that can be spent doing something else. That’s an argument for using default methods–lower opportunity costs. If we all tried to go back to first principles and reinvent the wheel all the time, we’d never get anywhere! On the other hand, thinking through one’s choice of method might have some benefits, regardless of whether one ends up choosing a good method. Presumably, if you’re going to be a good scientist, you need to get practice thinking through methodological choices.
Ok, I think that’s a much clearer set of options. So, tell me again: which one(s) worry you? 🙂
p.s. of course, in reality, methodological evaluation is multidimensional and holistic, so this whole conversation arguably is a bit misplaced. Arguably, a better conversation would be about the different dimensions on which methods can be evaluated, and whether ecologists tend to overrate the value of some dimensions relative to others.
LikeLiked by 1 person
So, turns out you didn’t stop blogging, you just now write entire blog posts and put them in my Replies 🙂
These are all interesting, but they miss a major dimension, I think – so I’d modify your second one. Often, a method is good in situation A. So it becomes the default for use in situation A, but also A’, and A”, and eventually B and C and D, for which it was never designed and doesn’t really work all that well…
LikeLiked by 1 person
“So, turns out you didn’t stop blogging, you just now write entire blog posts and put them in my Replies 🙂”
Heh. The same thought occurred to me as soon as I posted that comment. 🙂
Agree that your suggestion is an interesting variant on my second possibility.
LikeLiked by 1 person
The alternative to using methods just because everyone else uses them is to use methods that have been properly validated as fit for purpose, but that approach is rare in field biology.
LikeLike
I am just about to proudly submit a paper that doesn’t contain a single acronym or initialism! I did not only heroically avoid the trap of abbreviating a technical term that everyone else unnecessarily abbreviates. I also got somewhat fanatical and ditched my AIC comparisons and rephrased the methods so that I could get away with writing “generalized linear mixed models” only once. Sadly though, I did abbreviate scientific names because the journal doesn’t allow common names. 🙂
I think such efforts should be rewarded somehow. Now that Wiley has their open research batches, maybe there could also be little badges acknowledging papers that are free from self-invented abbreviations!
With another manuscript, I surrendered after a long internal fight and settled for SCP instead of supercooling point, which would have appeared at least five times in one paragraph and a lot elsewhere as well. Abbreviations are, after all, sometimes useful. But maybe we could, in such rare situations, help readers by using the unabbreviated versions as reminders in a few strategic places even after the first mention. At least, they probably won’t hurt in figure legends, and maybe they could occasionally be placed at the beginning of a paragraph, where it would be weird to use an abbreviation, as long as it’s obvious that it refers to the same thing as the abbreviation (and thus doesn’t confuse readers who have forgotten the unabbreviated version and just read the letters).
Your yeast example is enlightening. I have a hard time understanding how scientific writers keep using stupid abbreviations (like in one paper where the authors use the same initialism in parentheses to refer to two different things… and don’t even use those initialisms later in the text!). I jump between discussions and introductions to check abbreviations all the time. You’d think that people would hate doing that and wouldn’t want to force others to do it! I guess it has a lot to do with our familiarity with our subjects making us blind to the amount of information that’s new to our readers.
LikeLike
Zero acronyms is a real accomplishment! Glad you liked the post.
LikeLike
Pingback: P = 0.05 and a teaspoon of baking powder | Scientist Sees Squirrel