Have you ever read a scientific paper that simultaneously left you in deep admiration, but also crushed? I have – just now. It’s Rubega et al. 2021, “Assessment by audiences shows little effect of science communication training”. In a nutshell, several of the authors teach what sounds like an absolutely terrific graduate course in science communication; they used elegantly designed methods to test whether taking the course helps students do better at science communication; and much to their (and my) disappointment, they found that the answer was a pretty convincing “no”. To which I can really only say “argh”.
I won’t recapitulate the study in detail, because you should read the paper. You won’t regret doing so: it’s beautifully clear and eminently readable. And so clever! The authors measured communication effectiveness* for students before and after they took the course – and, elegantly, for matched “control” students who didn’t take the course. Both groups got better over time – but not by much, and the students in the course didn’t improve more than controls. Not only that: different observers showed remarkably little agreement in rating the effectiveness of any particular student. In other words: teaching science communication doesn’t make better science communicators, and in any event, different observers don’t even agree on which science communicators are better. As I say: “argh”.
This all raises some extremely interesting questions about science communication, and teaching, and talent. But before getting to those: can we celebrate the bravery of instructors who take rigorous steps to measure whether or not their course is effective? And, upon finding that it is (arguably**) not, who publish the results? I am somewhere between impressed and astounded.
OK, those interesting questions. First, what’s up with the lack of agreement in scoring? In the paper, different observers simply didn’t agree on which science communicators are better. Maybe I shouldn’t have been surprised. Some people idolize David Suzuki, while others can’t stand him; ditto for Neil deGrasse Tyson. Perhaps science communication is more art than science in the sense that there isn’t a single scale from worse to better. Perhaps people relate to science communicators on an idiosyncratic and personal level, and that relationship is more important than the number of jargon words per minute (or any other objectively measurable thing). Nobody expects us all to agree on whether Manet or Monet was a better painter; maybe nobody should expect us all to agree on whether Carl Sagan or Bill Nye was (is) a better science communicator.*** But surely we’d all agree on something about this? Surely the observer panel would all agree that Suzuki, deGrasse Tyson, Sagan, and Nye all communicate science better than me? You’d hope so – but the paper leaves me wondering.
Second, can you really not teach science communication? That a superbly designed, full-semester course in science communication doesn’t improve students science communication is nothing short of shocking to me.**** What are we to make of this? One possibility is that science communication can’t be taught because it isn’t a single thing. That’s the most nihilistic reading of the observers-don’t-agree thing from my last paragraph – but unless I really am just as good as Suzuki and deGrasse Tyson and Sagan and Nye, I don’t think we can end up there. The other possibility is that science communication can’t be taught because it’s an innate talent. Now that’s a possibility that will ruffle some feathers, because we really don’t like the idea that some talents aren’t teachable. It just doesn’t seem very egalitarian. Any of us should be capable of being Derek Jeter if we just spend 10,000 hours fielding ground balls at shortstop – or at least, that’s what we want to think.*****
What if there really are unteachable, innate talents, and science communication is one of them? What if Suzuki and deGrasse Tyson and Sagan and Nye are (were) simply gifted? I think one reason we rebel against claims like this is that they seem deterministic: no, you can’t be whatever you want to be, because there may be some things you’re just not good at. That seems limiting to some folks, shutting down avenues for people who should be free to do whatever they want to do. I actually find it the opposite. I’m a horrible artist – I can’t draw a recognizable stick person to save my life. My mother used to tell me that anyone could draw if they just tried hard enough. She apparently thought this would make me feel better. It made me feel much worse: I was failing at something I should be able to do, which meant either I wasn’t trying hard enough (lazy!) or I was trying ineffectually (stupid!). To me, the notion that maybe drawing just isn’t my talent is liberating. It doesn’t mean I can’t draw, if I enjoy it; it does mean I can stop beating myself up over the ugliness of the result.
OK, I got a bit distracted there by my unresolved childhood issues. Back to science communication. If some people are just innately talented, then is the job of SciComm training only to identify and encourage those people and let them build experience? I hope not. I hope the paper is wrong; I want us to produce many more Sagans and Suzukis, because science communication is incredibly important. We desperately need the general public to buy in to science. In the short run, we need them to buy in so they’ll choose to be vaccinated against Covid-19 (and other diseases). In the long run, we need them to buy in so we can maintain and expand the great program that is publicly-funded science as a public-benefiting good.
So I’d like us to train more Sagans and Suzukis, and so I hope Rubega et al. are wrong. I’m 100% certain that they’d like, and hope, the same things.
Image: Bill Nye and his bow tie © Ed Schipul via flickr.com, CC BY-SA 2.0
*^This is very hard to measure. Briefly: each science-communication student was asked to explain the scientific process to a journalism student, and a panel of other scorers rated video of that explanation for clarity, engagingness, and credibility. You might wonder why “the scientific process”, and not an actual piece of scientific content. I wondered that too. You should read the paper.
**^Arguably, I say, because there other possibilities: that the course is effective when measured on longer time scales, that the course would be effective if only the instructors were better, that the particular measurement of communication effectiveness isn’t appropriate, and so on. The authors discuss these with refreshing candour. You should read the paper.
***^Except that the correct answer is clearly Carl Sagan. In which resolute belief I completely undercut my own argument.
****^And it was clearly just as shocking to the authors – you don’t even have to read between the lines to figure that out. You should read the paper.
*****^That we can all agree that 10,000 hours of practice won’t make us all Shaquille O’Neal is irrefutable proof that basketball, as a sport, is intrinsically flawed. Until we move the baskets about 12 feet higher, it simply isn’t possible to be an elite centre without being tall – and that’s just dumb. And now, having terminally offended much of the free world, I return you to your regularly scheduled blog post. (Also, you should read the paper.)
Scientist Sees Squirrel 
Interesting reflection! Nevertheless, I think the main assumption of their study is flawed. When you teach complex skills, like those involved in professional science, you cannot expect immediate or isolated results. Good courses are very important for sure, but they are only the beginning. Their role is to point the students to previously ignored paths. Each student needs to decide which of those paths to actually walk. Someone interested in communicating science more effectively needs to take one or a few good courses and afterwards continue acquiring knowledge from many other sources. And, above all, to practice, practice, and practice, and stay open for critique, as well as new techniques and approaches. Therefore, the impact of a single science communication course, if measured alone, may seem small or null. The impact of a gate is to open a path, but the gate is not the path.
LikeLiked by 3 people
Interesting – I used to run a presentation skills course and that definitely improved performance, but whether this made the students better at presenting science to a non-science audience was not something I measured.
LikeLiked by 1 person
And in the greatest paradox of 2021 so far, a paper that shows that science communication cannot be taught, does show that it can be bought; it is paywalled.
LikeLiked by 1 person
This should solve that problem 🙂 margaret.rubega@uconn.edu
LikeLike
Such a complex topic. Science communication depends on abilities that may not be teachable in a single semester. Good science communicators (in written form) usually read a lot and read a lot of different topics. We all know great scientists, that read a lot of research papers and are not good to convey ideas clearly. In my experience, scientists that read a lot of literature and non-scientific media are usually better at communicating for non-scientists. Can a semester of science communication make up for a lack of communication abilities developed through basic educaction, access and incentive for literature reading and writing?
For spoken communication, I usually find social “skills” such as empathy important also. Some students just can’t put themselves in other people shoes so they fail to understand why other people would find this topic hard to understand, and they end up communicating as if others understand the subject as they do.
LikeLiked by 1 person
Interesting thoughts and I agree with some of the other commenters. Science communication is so multifaceted I think the answer would be yes, it is absolutely unteachable in one unit, even a single course. And I think any type of effective training in science communication depends on the student having some basic training in science first.
“Surely the observer panel would all agree that Suzuki, deGrasse Tyson, Sagan, and Nye all communicate science better than me?” No actually, I wouldn’t agree with that generalised statement. 🙂
It’s also true that science communicators are rarely ‘good’ at everything, and we shouldn’t expect them to be…they have skills in a particular field or a particular type of communication. I’m a writer, I’m not comfortable with communicating via performances, or live tv. And I certainly wouldn’t be good at communicating the latest physics news because I don’t understand it!
LikeLiked by 2 people
Just adding my agreement with Manu: I don’t think it’s at all obvious that Suzuki, deGrasse Tyson, Sagan, and Nye are all ‘better’ science communicators than you, Stephen. For the reason Manu identifies: just because you’re (say) good on live tv doesn’t mean you’re good in other media. And just because you’re good at communicating one sort of information to one sort of audience doesn’t mean you’re good at communicating other sorts of information to other audiences.
LikeLiked by 1 person
While I agree the question is a bit more nuanced than I had room for in the post, and while I appreciate the kind words, I’m going to double down. All four of those people, and many many others, are/were better science communicators than me. Which I’m completely OK with!
LikeLike
I think we need a poll on whether you’re a better science communicator than those other people. 🙂
LikeLike
Don’t make me TRIPLE down, Jeremy 🙂
LikeLike
All I’m saying is, science communication skills are best judged by the audience. You’re in no position to judge your own skills, buddy. 🙂
LikeLiked by 1 person
Maybe trying to turn everyone into a science celebrity is both uneccesary and way too ambitious. Just getting everyone up to a level where they don’t deliberately obscure their message (or that they have no message) with academic-speak and smoke screens of “modelling” would be a more achievable aim for a one-semester course.
LikeLike
Agreed – but the paper indicates that that aim is not, in fact, achievable in a one-semester course!
LikeLike
Is the measurement issue here because all of the students were decent at communicating science? So that the differences between them were fairly small? Hence the disagreement among the evaluators?
Maybe the glass-half-full interpretation here is that the students involved in this study came in as decent communicators. They’d probably all had some practice in the past at giving presentations to classmates, for instance. They probably all like science, and like talking about science. Etc. Once students have reached that level, maybe further improvement just gets much harder.
LikeLiked by 1 person
Just to undermine my own comment a bit: there are other contexts in science (and life) in which there *is* appreciable agreement among evaluators, even though there’s not *that* much variation in “quality” among the entities being evaluated. Peer review of papers and grant applications, for instance. Or for a non-scientific example, think of judging Olympic figure skaters and gymnasts. So if there’s literally *no* agreement among evaluators of science communication, then yeah, that is worrisome.
LikeLiked by 1 person
Unless you know of other studies, agreement between peer reviewers is surprisingly poor. According to Welch’s 2012 preprint “Referee Recommendations” (search SSRN for it; if I incude the URL, my comment is rejected) only one third of reviewer judgement seemed to be true signal for the quality of the reviewed paper; one third was the general niceness of the reviewer, and the last third was sheer random noise.
LikeLiked by 1 person
I’ll try again: it’s https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2137119
LikeLike
https://onlinelibrary.wiley.com/doi/pdfdirect/10.1002/ece3.4467
LikeLike
Interesting study, but measuring something very different from what interests me, which is how *good* the paper is. How sexy it is may also be of interest, of course, but knowing that reviewers are pretty good at guessing that doesn’t reassure me much. (And as someone who by policy never submits to “selective” journals it doesn’t directly affect me.)
LikeLike
Unfortunately, my institution doesn’t have access to the full article.
I think the issue is that for effective science communication training it needs to be more than just a semester. As someone who has been fortunate enough to take classes at the Alda Center and have experienced lots of peers, I think the skills just need more time to be developed. I also have noticed that people need to be taught how to communicate better in general before even delving into how to communicate science better. People are always too stuck on talking but talking isn’t communication. People need to learn self awareness so they can identify where their communication is lacking, they need to listen, they need to pick up on their audience’s nonverbal cues, they need understand their emotions and the audience’s emotions, and be present not practiced, etc. So do I think science communication is an innate skill, no. What I do think is that self awareness precedes good communication which precedes good science communication and I think that takes a whole hell of a lot longer than a semester to work on, maybe even a lifetime.
LikeLiked by 1 person
Your institution almost certainly has a document delivery service (might be called “interlibrary loan”. Ask a librarian! But more importantly: sure, it’s possible that a 6-semester course might have detectable effects. But we have no data suggesting that that’s true, and I wonder why we should default to that belief – other than that all of us, including me, would very much like it to be so?
LikeLike
In simple terms it’s all about looking smart to your colleagues and professors. You want them to be impressed with your command of the English language. If you do aim to make your paper more easily understood, you see it as a risk of not being seen as a serious scholar.
To learn more about that skill, I would recommend becoming a referee/ reviewer for a journal. If you must make sense of the manuscript, you will likely realize how over worded most are. You are allowed to make criticism of wording that leaves the reader confused. Once you have done that for a few months or a year, you will realize how bad some of your manuscripts have been.
Remember that a manuscript is an effort to clearly communicate an idea or finding. Too often the sections sound like a contest for how many esoteric words can be used. A manuscript is not like a term paper that must have a minimum number of words.
The idea is to have the manuscript flow smoothly from the question to the answer to that question. Believe it or not, many manuscripts never answer directly and clearly the question posed in the beginning of the manuscript. The reader does not want to engage in an archeological dig to find the answer. What might have been an important finding may end up as rejected because the writing left the reviewer/ editor baffled (it really happens).
A really good manuscript is like telling a story that has a premise and a clear, succinct end. Think about Little Red Riding Hood, and how the story moves along without specifying the breed of wolf, the ailments of grandma, how old Riding Hood is and whether her parents should have denied her request to visit grandma, and so forth. Or, as Sergeant Friday from Dragnet used to say: “Just the facts, man”.
LikeLiked by 1 person
Richard, did you mean to leave this comment on a different post?
LikeLike
I was speaking to the inability of research types to communicate clearly in journal articles or reports to management. I think that some simple principles of good writing can be expressed to a SciComm class. But it is my belief, after 40+ years as a statistician in medical research, that even super smart people are not used to writing for clarity. Rather, they are used to reporting results like we did when writing up the results of a textbook chemistry experiment. Worse, in math and stat classes, you simply circled the answer, and that’s all that was needed. So long as the bare facts were there in a report, that was good enough for schoolwork.
What students rarely appreciate in the first years out of college or grad school is that there is a silent but real change in the rules of report writing when you cross the stage to get your diploma. Experience, laced with sarcasm and criticism, is often the most effective teacher of how to write.
LikeLike
No one should be saying their training improved communicating skills without using the appropriate audience as reviewer. When you do that, it appears that training does little but improve the communicator’s personal impression, probably bc they are used to being in front of an audience (when speaking). The trainees always say they improved, and so do the trainers. That is barely relevant. It’s the audience that counts.
LikeLiked by 1 person
You got that right!
You can be the best theorem solver or score the highest on the qualifying exam. Your thesis may break important new ground. You might be the shining star of your graduate program.
But your professional success will mainly depend upon your ability to communicate with the audience at the place where you work.
You are hired by a company to help management make decisions that they can rely on. When presenting results to management either orally or in writing, you have to be able to express those findings with language and flow that even “Aunt Minnie” could follow. If they don’t “get it”, then you are of little or no value to the company. Bye-bye.
If you are not sure of the math abilities of your audience, you should not be ashamed to use trivial examples that help the audience understand how a difference compares to a ratio, to a relative difference, and so forth. If you overestimate your audience’s comprehension of math and other technical stuff, your marvelous presentation will leave the audience saying: “Now, what did he say??”
LikeLiked by 1 person
I wonder if we are focusing on the wrong part. Your post and a lot of the comments focus on the communication part of science communication. What the issue is science itself? Maybe it is the research we do, the science we create, that makes it difficult to communicate? I am curious what would happen if Rubega et al. 2021 used their course for communication with a different focus. Would they have the same results? It might be fun to flip this experiment on it’s head and see if different fields of science are easier to communicate. If we could identify what makes certain science easier to communicate, we could then take those characteristics and apply them elsewhere. Some food for thought. Thanks as always for the thought provoking post.
LikeLiked by 1 person
this led me to think that maybe a 1 semester science communication course does not make us a better communicator, but makes us a better scientists in skill and perceptions unrelated to being better at communicating science. I feel that other aspect researchers could evaluate in the future is better understanding of the scientific endeavor, of bias and society impact of the research, of history of science and so on
LikeLike
Pingback: Friday links: remembering David Schindler, outsourcing peer review, science communication is unteachable (?), and more | Dynamic Ecology
Pingback: Yes, that paper is paywalled. But you can read it anyway. | Scientist Sees Squirrel
Pingback: Don’t use your Discussion to shred your own work | Scientist Sees Squirrel