Turning our scientific lens on our scientific enterprise: a randomized experiment on double-blinding at Functional Ecology

Image: Experiment, © Nick Youngson via picpedia.org, CC BY-SA 3.0

I’m often puzzled by the reluctance of scientists to think scientifically and do science.  “Wait”, you say, “that’s a bizarre claim – we do science all the time, that’s why we’re called scientists”.  Well, yes, and no.

We love doing science on nature – the observations and experiments and theoretical work we deploy in discovering how the universe works.  What we don’t seem to love nearly as much is doing science on ourselves.

Of course you can do significance testing on simulation data!

Warning: wonkish. Also long (but there’s a handy jump).

Over the course of a career, you become accustomed to reviewers raising strange objections to your work.  As sample size builds, though, a few strange objections come up repeatedly – and that’s interesting.  Today: the bizarre notion that one shouldn’t do significance testing with simulation data.

Yes, most reviews are submitted at the deadline. No, that doesn’t justify shorter deadlines

Image: Deadline, by geralt CC 0 via pixabay.com.

Warning: I’m a bit grumpy today.

I’m back tilting at one of my favourite windmills today: requests for manuscript reviews with unreasonably short deadlines.  I’ve explained elsewhere that one should expect the process of peer review to take a while.  Journals would love to compress the process by reducing the time the manuscript spends on the reviewer’s desk – and so they ask for reviews to be returned in 2 weeks, or in 10 days, or less.  As a reviewer, I don’t play this game any more: I simply refuse all requests with deadlines shorter than 3 weeks.

I’ve asked a few editors and journal offices why they give such short deadlines, and they give two kinds of answers: one outcome-based, and one process-based.

I refuse all review requests with deadlines < 3 weeks. Here’s why, and how.

Warning: another grumpy one

I’m seeing it more and more: requests to review manuscripts with ludicrously short deadlines.  Sometimes 10 days, sometimes 7, sometimes one week (5 business days).  And I see editors on Twitter bragging about a paper they’ve shepherd through the entire review process in 5 days, or a week, or two weeks.  I want all this to stop.

The efficiency of the lazy scientist

Photo: Lazy red panda CC 0 via pxhere.com

I’ve just published a paper that had some trouble getting through peer review.  Nothing terribly unusual about that, of course, and the paper is better for its birthing pains.  But one reviewer comment (made independently, actually, by several different reviewers) really bugged me.  It revealed some fuzzy thinking that’s all too common amongst ecologists, having to do with the value of quick-and-dirty methods.  Quick-and-dirty methods deserve more respect.  I’ll explain using my particular paper as an example, first, and then provide a general analysis.

Why journals like “reject, but resubmit”

It happened to me again, a few weeks ago: a manuscript I’d had high hopes for came back from the journal with a decision of “reject, but with an invitation to resubmit”.  It’s better than a flat-out reject, to be sure, but disappointing nonetheless.

There’s a widespread belief – almost a conspiracy theory – that journals use “reject, but resubmit” as a device to cheat on their handling time statistics (by which we mostly mean time from submission to first acceptance).  After all, if a manuscript gets “revision”, the clock keeps ticking from the original submission; but “reject, but resubmit” means we can pretend the resubmission is a brand new manuscript and start the clock over.  Clever but deceptive move, right? 

Four unconvincing reasons not to crowdfund science

Image:  Crowdfunding, US Securities and Exchange Commission (no, really), CC BY-NC-SA 2.0.

Sometimes I hold an opinion that I’m almost certain has to be wrong, but I can’t figure out why. This is one of those times.  I need you to help me.

I’ve been watching the trend to crowdfunded science, and it bothers me.  I completely understand why it happens, and why it’s become much more common. The science funding environment continues to be difficult – indeed, in many places it seems to be getting steadily more difficult, especially for early-career scientists and those doing the most basic/curiosity-driven science.  At the same time, the rise of web-based crowdfunding platforms* has made it relatively easy to reach potential donors (at least in principle, and more about that below). For any given researcher at any given time, surely the science is better with access to crowdsourced support than it would be without.  And several colleagues I like and respect have crowdsourced part of their work.  So why am I so uncomfortable with the model?

How good (a manuscript) is good enough?

Image: © (claimed) Terrance Heath, CC BY-NC 2.0

“How good a manuscript”, I’m sometimes asked, “is good enough to submit”?  It’s a natural enough question.  A manuscript heading for peer review isn’t the finished product.  It’s virtually certain that reviewers will ask for changes, often very substantial ones – so why waste time perfecting material that’s going to end up in the wastebasket anyway?

“Peer Community In”: Beyond the traditional publishing model (guest post)

I recently learned about Peer Community In (PCI), a new system for reviewing and recommending preprints. I’m really intrigued.  It’s true that I’m an old fuddy-duddy who’s on record as saying that we often exaggerate the problems with the status quo, and as not liking to think outside the box.  And yet there are good reasons to think it might be good to have other ways beyond traditional journals to disseminate science.  We should experiment with a variety of new systems, and PCI seems like one well worth exploring.  Read on to learn more!

What follows is a guest post by Denis Bourguet (denis.bourguet@inra.fr), Benoit Facon (benoit.facon@inra.fr), Thomas Guillemaud (thomas.guillemaud@inra.fr), and Ruth Hufbauer (hufbauer@colostate.edu).  DB, BF, and TG are the founders of PCI, and RH is a colleague and member of the board of PCI Evol Biol.

We believe that the current system of publishing with academic journals suffers from four crucial problems. First,

Persistence in publishing: the Tubthumping strategy

Image: Asim Saeed via flickr.com CC-BY-2.0

 This is a joint post by Steve Heard and Andrew Hendry (crossposted here on Andrew’s blog). 

Another week, another rejection, right?  If you’ve been in science long at all, you almost certainly have a bulging file of rejections for grants, manuscripts, fellowships, and even jobs. Here, for example, is Steve’s truly impressive job-rejection history; and here’s a previous analysis of Andrew’s manuscript rejections.

We were part of a recent Twitter exchange that began when Steve tweeted in celebration of submitting a manuscript – to its third different journal: