Photos: “Mentor”, from my cherished Oxford Universal Dictionary (1933; 1955 reprint). As a very small child, I sat on this dictionary at my grandmother’s dinner table. When I no longer needed a boost in height, it provided a boost to my vocabulary instead. Yes, I’m a nerd. Longtail widowbird: Public Domain by Mohanr53 via wikimedia.org.
It’s grant reviewing season*, and that’s gotten me thinking about mentorship. NSERC (Canada’s main basic-science granting agency) refers to the students and employees a PI supervises as HQP (for highly qualified personnel), and weights both past HQP training record and future HQP training plan very heavily in its deliberations**. Although NSERC funding rates are relatively high (and the grants correspondingly small, and that’s good), they’ve been tightening somewhat, and this seems to be driving some remarkable supervisory inflation.
What do I mean by “supervisory inflation”? In the grants I review, I’ve seen the number of HQP reported as “trained” or “mentored”, and also the number promised for future training, increase year over year. In the last few years, I’ve seen multiple grants claim mentorship of 50 or more (in one case, considerably more) HQP over a 5-year granting cycle. This implies somewhere between 15 and 25 HQP in the lab at any one time, depending on the blend of grad students, short-term field assistants, and so on***. That’s a lot, at least in my field of ecology and evolution, and I have trouble taking such claims seriously.
The problem? I just don’t see how a PI can deliver quality mentorship to 20 HQP at a time. Imagine quite conservatively that you spend just 5 min a day checking in on what each is doing, with a more substantive 30 min meeting every other week. Then assume your average HQP produces three document drafts per year, and you spend 3 hours reading and commenting on each. (These drafts might include manuscripts, thesis chapters, proposals, and so on; and while field assistants may not produce any, postdocs and senior grad students will shower you with them.) A little arithmetic suggests you’ve committed 17 hours/week already, and we haven’t included much more than basic supervision and a bit of editing help – a rather minimalist definition of “mentorship”****! That’s more than a third of a 50-hour workweek (few of us sustain more than that, even if we pretend to, and we shouldn’t) – and that workweek also has to pack in teaching, service, and our own contributions to research!
My calculations suggest that if you really intend to have 50 HQP during your 5-year NSERC grant, you’re planning to give each one rather little attention. There are two ways to do that. One is to take on students (and other HQP) who are a normal cross-section of the population, and let them flounder unassisted. The other is to take on only the most outstanding students and employees – the ones who are thoroughly prepared, brilliant, self-motivated, self-directed, and self-correcting, and need only to be issued keys to the lab. These ones you can ignore, until they come to you with a completed thesis or a job all done – but if you can fill a lab entirely with people like this, you’re an unusual bird indeed.
Actually, if you could fill your lab with HQP who don’t need mentoring, would you want to? I don’t think I would. Over my career I’ve supervised all kinds of students and employees, including a few who would have excelled with or without me, and a few I poured heart and soul into. I enjoyed both, but I’m only proud of the latter. It’s tremendously satisfying to take a poorly prepared undergrad into the lab, bring them along and send them off to grad school thinking and acting like a scientist; or to take on a new grad student whose admission you have to argue for, and then to nurture their spark and see the fire slowly spread. That’s what mentoring is – helping someone fulfill potential that they aren’t able to fulfill on their own. You can think of this as taking satisfaction in value added by mentoring (to borrow some lingo from economics).
I started with my irritation at supervisory inflation in grants, so let’s get back to that. I understand why people make ludicrous HQP claims: granting agencies like NSERC reward them for doing so**. Any system produces exactly the behaviour it incentivizes: PIs are driven by NSERC’s criteria to stock up on HQP (real or claimed) just as the male longtail widowbird is driven by female preference to carry a tail it can barely fly with*****. The HQP incentive structure was always guaranteed to drive an arms race; and I’ll admit (with a little discomfort) that when my last proposal was due, I didn’t miss many opportunities to load up my list of mentored HQP. But: the system isn’t doing anybody (mentors or mentees) any favours, as it seems guaranteed to produce either deceptive proposal-writing or superficial supervision by harried PIs.
The solution to supervisory inflation is simple in principle, of course (although admittedly I have no idea how to work this in practice): agencies should incentivize value-added mentoring. Just as I take satisfaction as a mentor for the value I’m able to add, when I submit a grant, I should receive credit only for that value added. Taking a dozen superbly prepared elite-scholarship students who defend brilliantly written theses and waltz into Ivy League professorships should count for next to nothing. Taking just one or two students who barely scrape the admission bar, and turning them into able researchers ready to take their next steps, should count for a lot.
Now that I’ve written this out, I see something I didn’t see when I started (this often happens when I write a post). I’m nearly always disappointed with HQP training plans (including my own) because they’re 99% boilerplate and motherhood-and-apple-pie statements. Now I know how the next one I write is going to stand out: I’m going to explicitly propose value-added mentoring. I’m going to propose deliberately taking fewer students and less prepared ones, and work harder at mentoring each. This may swim against the current, but sometimes that’s what one should do.
© Stephen Heard (email@example.com) January 18, 2016
Related post: Why grant funding should be spread thinly
*Much like five o’clock, it’s always grant reviewing season somewhere. But here in Canada, NSERC Discovery Grants are dished out to reviewers in late fall, and considered by panels in January.
**NSERC Discovery grant review gives equal 1/3 weight to “Scientific excellence of the researcher”, “Merit of the proposal”, and “Contribution to training of highly qualified personnel”. Other agencies, including NSF, also place considerable weight on contributions to training, although I don’t know how it’s formalized.
***50 distinct individuals over 5 years isn’t 10 per year, because many individuals represent multiple personnel-years – a PhD student, for example, will typically stay in the lab 4-5 years.
****Yes, I’m aware that one can have senior HQP help mentor junior ones, reducing PI time investment per HQP. At least for me, though, I’m not sure it’s plausible or ethical to reduce it much from my baseline calculation. And in any event, the kind of HQP who can take on some of the mentoring – postdocs, long-term lab techs, and so on – are too expensive to be supported on the very grants I’m seeing suffer HQP inflation.
*****I bet you were wondering what the widowbird had to do with all this.
Glad to see you bring this up Steve. While I see this as a problem, I hope that reviewers are able to see through the spin, and do more than just add up the numbers.
I suspect (hope) that this is the case based on the n=1 of my NSERC application from last year. One of the achievements I was most proud of was that 4 of my undergraduate students had published elements of their undergraduate theses (in two cases with sufficient contribution to the final paper be the lead authors). Taking the time to mentor undergraduate writing takes a lot of time, and hence limits the number of students. I was quite pleased to note that the reviewers of my application took the time to highlight this as a significant positive. I hope everyone else reviewing similar cases does the same.
Stephen, how are people actually inflating these numbers? Because on NSERC grants, you usually have to be specific enough regarding HQP that I would think it would be kind of difficult to inflate. You have to name the HQP, you have to name their project, etc. Are people claiming inflated credit for jointly-supervised or jointly-supported HQP? (if so, yeah, that’s pushing it) Did people not used to claim undergraduate honors students as HQP, and now they do? (if so, that seems totally appropriate to me) Or what?
And re: the implication that people training lots of HQP aren’t giving them much individual attention, on an NSERC grant that’s where the HQP training plan comes in. If somebody’s HQP proposed training plan conflicts with the number of HQP they propose to train, then they ought to be dinged for that. So what are people who are claiming implausibly-large numbers of HQP saying in their training plans?
I think there’s lots of reporting of jointly-trained or only peripherally supervised students. You are right, you (sort of) have to report names and projects, but you can get away with not much detail on that end. But I also see just totally implausible claims in HQP training plan sections – I’ll see someone saying that they meet with every student regularly, but there are so many HQP claimed that you couldn’t do that (well, not that and everything else). Yes, I agree, these folks ought to be “dinged”!
re: HQP who don’t “need” that much training: do you give PI’s credit for attracting really strong graduate students (major NSERC fellowship holders, say)?
Re: ** : with NSF, training is lumped in under “broader impacts,” along with everything else that might be a broader impact. As far as I know, you don’t *have* to talk about mentoring anyone as long as the rest of your broader impacts are stellar. But it’s a pretty easy thing to put in there, and I think most people include it. Of course, broader impacts appears to also be an arms race (from my admittedly limited experience). It seems to me that it’s no longer okay to just mentor students. Now you have to come up with novel ways to mentor students or new programs and experiences that mentees can access. For postdoc mentoring, NSF requires a “postdoc mentoring plan,” but as far as I can tell, no one really cares about it as long as you include all the reasonable boilerplate stuff. (And as far as I know, there’s no follow-up to make sure the things in the plan actually happened.) So sounds similar to the Canadian system in that regard.
Pingback: Friday links: fitness vs. #365papers, and more | Dynamic Ecology
When NSERC changed its rules a few years ago to make the HQP body-count one of its evaluation criteria, this type of gaming was the entirely predictable result. A highly accomplished younger colleague simply advised me to put my best foot forward, as in, nudge-nudge-wink-wink, make stuff up. (Although in his/her case, I’m confident that he/she didn’t need to shade the truth. But if that level of cynicism is at all common, it’s hardly a major advance for the well-being of Canadian science.) For example, if I felt that I was doing a particularly diligent job of participating in a graduate advisory committee, I should elevate that to calling myself a co-supervisor. On balance, I’m just glad to be close enough to retirement that I could afford to let this stuff go.
Great post, thanks for bringing attention to this issue.