Given how much time and energy we academics put into evaluating each others’ CVs, it’s a bit startling to realize that we’re doing it wrong. Hiring decisions, promotion decisions, tenure decisions, grant funding decisions – all of these draw heavily on the candidates’ records of “excellence” as documented on their CVs. There’s nothing wrong with that, of course – there’s no other way science could be the meritocracy we like to think it is, and that it ought to be (but isn’t yet). But nearly every time I’ve been involved with evaluating CVs, the process has involved an enormous mistake: we’ve* paid attention to the candidates’ past records of grant funding.
There’s a very simple reason why grant funding should receive absolutely no attention in our assessments of each other:unlike everything else we assess, grant funding isn’t a scientific output – it’s a scientific input. My own tenure assessment, for example, considered how many courses I’d taught (and how well), how many papers I’d published (and how much impact they’d had), how many undergrad and grad students I’d trained, and a bunch of other things. All of these are outputs: aspects of my contribution to the accumulation and dissemination of scientific knowledge. But it also considered my record of grant funding, and that was bizarre – because just like the square footage of my lab and the number of hours I spent at work, grant funding is an input. You don’t judge a cabinetmaker by how many tools they own – you judge them by the cabinets they’ve made.
It doesn’t make someone a better or more productive scientist to have more grant funding. In fact, it’s arguably the reverse. Let’s do a quick thought experiment. Over the last 5 years, Scientist A has published 10 papers and garnered $1 million in grant funding. Over the same period, Scientist B has published 10 papers but received only $100,000 in grant funding. Which scientist has the better record? You’re right: hands down, it’s Scientist B. Scientist B has produced the same scientific output with 1/10 the funding input. Scientist B has shown their ability to remain productive through the I’m-between-grants penny-pinching periods that sprinkle most careers. Scientist B has consumed fewer of the resources that go into adjudicating, awarding, and administering grants. Which scientist would you rather have as a colleague? For me, it’s Scientist B, every time. But it’s Scientist A the academic world seems to admire.
I’ve sat on at least one grant evaluation panel that has explicitly argued that it’s not the grant funding itself we’re considering – instead, we’re using that as evidence that other agencies have seen excellence in the candidate. This makes a lot of sense – but only if you don’t think about it much. If you do think about it (and aren’t academics supposed to do that?), you’ll likely conclude that at best we’re ducking our own responsibility to make a judgement. At worst, we’re being actively harmful. When we hire, promote, and especially fund people on the basis of the grant funding they’ve previously been awarded, we engage in a positive-feedback cycle that magnifies disparities among us. Scientist A, with their million-dollar grant history, is more likely to get the next grant, so the initially-rich get richer and the initially-poor get locked out of the system. This reduces society’s return on its investment in science, because we’ll get more done if funding is spread more evenly. It also sets up winners and losers by amplifying initial variation among researchers that may be trivial (because at most agencies, many fundable grants go unfunded) or, worse, result in part from unconscious biases. Considering previous granting success when we evaluate researchers gives our scientific community the same mathematical property that makes the board game Monopoly so thoroughly unenjoyable:** small early fluctuations in luck become magnified by positive feedback into advantages impossible for trailing players to overcome. Please, can we make science less like Monopoly?
When I came up for tenure at my first job, I was very naïve about all this. I had a substantial theoretical component to my work, and in my tenure package I argued that this was a strength, because it meant I was well positioned to weather any periods in which I was unable to secure grant funding. My department thought about this quite differently: it suggested to them I wasn’t sufficiently motivated to chase dollars. I was naïve, of course. Not because I didn’t understand the institutional thirst for money; of course I did.*** I was naïve because I thought that tenure committees, Chairs, and Deans would push back against it, and evaluate my scientific output. Some do push back, of course – but not as many as you’d hope.
Some decisions about how we should operate, as a community of scientists, are hard. This one is easy. No, this one should be easy; but I guess it isn’t, or we wouldn’t still be evaluating researchers based on how many dollars they’ve spent instead of how much good science they’ve done. Can we please stop?
© Stephen Heard November 16, 2021
Image: Cabinet, © Thomas Quine via flickr.com CC BY 2.0. Sure, it’s nice, but I hear the cabinetmaker only owned one hammer.
*^For me, the “we” here has been university hiring committees, university promotion and tenure committees, and funding agency proposal evaluation committees. I should make clear I’m not speaking for any of these bodies or for any of the organizations that set them up. Actually, given that the whole point of this post is that these organizations are doing it wrong, I guess that was going to become pretty clear shortly anyway.
**^For people older than 10, I mean. I will admit to having loved Monopoly as a child. Of course, I loved a lot of things as a child that shouldn’t be designed into our systems for doing science.
***^This was in the US, where the system of “overhead” on grants makes this institutional thirst especially acute. But the problem isn’t limited to the US. By the way: I was awarded tenure at that institution, despite the best efforts of departmental leadership, but I left shortly thereafter.