I’ve seen half a dozen posts and essays arguing that we should stop publicizing, listing, or paying attention to the names of the journals our papers are published in. The argument goes along these lines*. First, we should judge the worth of papers based on their content, not based on where they were published. Second, when filtering papers – deciding which ones to read – we should filter them based on what they’re about (as communicated by their titles and abstracts), not by the journal they’re in.
This argument is, I think, a logical extension of arguments against the impact factor. I think those arguments are overdone, and I think this one is too. When I look at paper lists on CVs, web pages, online alerts, and the like, I look for journal names and pay attention to them. I think it’s entirely reasonable for others to do the same, so on my own web site, I still use a conventional bibliographic format that includes the journal name for each paper.
I actually agree with the first half of the argument; it’s the second half I don’t buy. Yes, when reading papers (to cite them, in a tenure assessment, or whatever), we should judge them by their content and nothing more. But when deciding which papers to read, we need help. The deluge of published papers grows every year; not only can I not read all the papers relevant to my field, I can’t even read all the abstracts relevant to my field. I’m not sure I can even read all the titles relevant to my field – at least, not without becoming more of a narrow specialist than I’m comfortable with. If I pay more attention to the title of a paper in the American Naturalist than I do to one in Entomologia Experimentalis et Applicata**, it’s not (I claim) because I’m a journal snob – it’s because I know that journals have scopes and personalities, and some are more likely than others to print papers that I’ll find interesting and important.
Imagine, for example, that I see a paper titled “Long-term trends in population dynamics of the vermilion leafhopper under biological control”. That might interest me – I don’t work on the vermilion leafhopper***, but I’m generally interested in insect population dynamics. But the set of papers that might interest me is huge. Can I get a little more information that might predict the paper’s “character”? Sure.
- If it appeared in Nature, it’s probably flashy but unimportant. (This may not be true in all fields, but in ecology and evolution Nature seems to have more to do with biggest, first, longest, or just plain coolest than with most important. And I swear, this is not (just) sour grapes for never having gotten a paper through review there…
- If it appeared in the American Naturalist, it probably makes some kind of conceptual or synthetic advance that means it will have implications for population dynamics more generally.
- If it appeared in Entomologia Experimentalis et Applicata, it’s probably a narrower paper that will matter to leafhopper biologists and biocontrol practitioners, but not as much to me.
This kind of pre-assessment won’t work all the time, and it’s about character, not quality. If I could read every paper, or even every paper’s abstract, would I make better decisions? Sure – but I can’t.
Furthermore, even if I can’t judge an individual paper by the journal it’s in, I can get a good idea of a researcher’s approach and interests from the list of journals their overall oeuvre is in. If I’m advising an undergrad looking for grad programs (let’s say), and I don’t know the proposed supervisor’s work, I think I can make useful deductions from the array of journal names – deductions that take me beyond those I can make from the paper titles alone. Is every paper in Nature? Some eye-rolling may be appropriate. Is every paper in EEA? There’s a hard-core entomologist, the right supervisor for someone who adores insects above all else. A mix of Am Nat and EEA (etc.)? That’s likely a research program with more breadth, good for someone who sees insects as model systems for more general ecoevolutionary questions. I’m sure you could make similar deductions yourself, especially in your own subfield, and I’m not sure what you’d gain from closing your eyes to them.
I understand the desire to ignore journal names. It feels enticingly egalitarian, with the admirable ideal of letting scientific quality speak for itself. But in practice, I don’t think it’s likely to work. I’m happy to let our peer review and publication system do one of the things it does quite well: sort papers into bins that signal (even if imperfectly) to readers whether they might be interested in a paper, and why.
© Stephen Heard (sheard@unb.ca) August 29, 2016
Thanks to Timothée Poisot for spurring this post with one of his own , which provides an excellent counterargument to mine. You should read both posts before deciding whether you agree with me.
*^This argument is loosely paraphrased from Timothée Poisot’s blog post on the subject.
**^EEA is a perfectly good journal, and I’ve published in it several times. But even I would argue that my EEA papers are less important overall, and likely to interest many fewer readers, than my Am Nat ones.
***^Which is a good thing, because I made it up.
Scientist Sees Squirrel 
Hi – I don’t allow apps to follow people for me so I reply here. Your article makes me think of the pressure on academics to publish QUANTITY of papers
LikeLike
I confess I find it a little sad that this needed saying*, but well said.
I’d add another reason to care about publication venue, in the context of evaluating applicants for faculty positions with a research component: if you publish in selective journals, that shows that you can convince a careful, critical, independent expert** in your field that your work is interesting and important. Which is evidence that your work is interesting and important. It also demonstrates your ability to convince others of the interest and importance of your work, which you need to do to get grants.
*Though I’m not sure how many people would disagree with it. Many people who are vocal about this issue online would disagree–but I wonder if there’s a silent majority who agree.
**Cue people reporting anecdotes about how sloppy some reviewer for Nature once was, as if that refutes my claim here. Funny how nobody ever shares anecdotes about how sloppy post-publication readers often are…
LikeLike
Jeremy, re the selective-journal-publisher point: what about the converse? What if I choose to be non-selective (which I might justifiably do by going to Scientific Reports, PLOS One, the PeerJ) because I don’t want to waste time jumping through all the hoops that a selective journal inevitably requires? Not having gone the selective root doesn’t mean an author hasn’t demonstrated an ability to convince other people the work is useful or important (to an extent citations are a measure of that, but as you say, people are sloppy and may cite a paper just because it says Thing A which supports my Thing B). So how do you evaluate that or at least incorporate that into a fair evaluation?
I struggle with this, not because I care where a paper is published but because I find it difficult to reconcile what I personally think we should do (there should be no journals [plural] and we use a range of curation tools to find stuff to read) with my responsibilities as a mentor / advisor to students or post-docs. If you’re evaluating this way, I’d be doing them a disservice to be non-selective in journal choice.
LikeLike
“Not having gone the selective root doesn’t mean an author hasn’t demonstrated an ability to convince other people the work is useful or important (to an extent citations are a measure of that, but as you say, people are sloppy and may cite a paper just because it says Thing A which supports my Thing B). So how do you evaluate that or at least incorporate that into a fair evaluation?”
I have an old post on this (and much more):
https://dynamicecology.wordpress.com/2013/09/19/advice-how-north-american-faculty-position-search-committees-work/
One way to think about the answer to your question is in terms of lines of evidence. If you choose to only publish in non-selective journals, you’re forgoing one line of evidence among others as to the interest & importance of your work, and as to your ability to convince others of same. You’re choosing to rely on other lines of evidence, like the testimony of your references. Which, yeah, could be risky.
I also have an old post on how I decide where to submit my papers, which prompted some contrasting advice from Ethan White and Tim Poisot:
https://dynamicecology.wordpress.com/2013/01/24/advice-how-to-decide-where-to-submit-your-paper/
“I struggle with this, not because I care where a paper is published but because I find it difficult to reconcile what I personally think we should do (there should be no journals [plural] and we use a range of curation tools to find stuff to read) with my responsibilities as a mentor / advisor to students or post-docs. If you’re evaluating this way, I’d be doing them a disservice to be non-selective in journal choice.”
I’m probably a bad person to ask how to square this circle, since I like having selective journals. In general, it’s probably not a great idea to ask someone who doesn’t want to achieve X how best to achieve X, or how to achieve X while minimizing personal risk. 🙂 But FWIW, I guess I’d ask whether achieving the world you personally consider ideal (a world without journals) necessarily obliges you to choose all your actions so as to bring that ideal closer. I mean, aren’t there lots of ways to work towards that ideal that don’t involve one’s (or one’s mentees) choice of journal to submit to? And if it feels like a compromise of your principles to submit to journals that you wish didn’t exist, or to advise someone else to do so, well, I guess I’d say that life is full of compromises, and a journey of a thousand miles starts with a single step. In the long run a lot of radical reforms and massive improvements in all walks of life have come about as the cumulative effect of incremental (and so necessarily imperfect, n-th best) reforms, implemented by imperfect individuals making compromises unpalatable to idealists. And make no mistake, a world without journals would be a very radical reform of the scientific publishing ecosystem!
I guess I’d also suggest thinking about why you want a world without journals, and if there are other ways to achieve at least some of the same effects. I’m thinking for instance of the various ways in which one might address concerns about access to the content of subscription journals: via posting of preprints, via free access after an embargo period, via open access journals…
Just my two cents, obviously, your mileage may vary (temperamentally, I’m very much an evolutionary, not a revolutionary, if you couldn’t tell!). It’s always a very personal decision to decide how much or what you’re willing to risk or sacrifice for the sake of what you think is right.
LikeLike
(Sorry for the slow response Jeremy!)
Thanks for the reply and the links – I now have them saved to discuss in a future lab meeting.
I liked the response to my second point, the “personal beliefs vs mentoring role” part. I’ve seen Ethan White, for example, working along similar lines as you suggest: free-to-read as a Reports paper in Ecology if can’t do OA; preprints, especially if work ends up non-OA. There are lots of ways to be open and a little bit open is better than no open at all. As I’m running a couple of classes for our new grad students on Open Science in a few weeks I need to make a bigger play of these practical Open strategies to balance the advocacy side.
I will note that my utopian vision of one journal to rule them all I’m not solely or even mainly wishing that upons us from the point of view of “access”. I think we’d all be better off in many ways if where you published just wasn’t a thing we had to worry about, at all, for anything. This seems to be the source of a lot of distortion and ensuing problems in science.
LikeLike
I’m puzzled by Tim’s notion that deciding what to read by looking at the paper title and deciding what to read by looking at the publication venue are substitutes. Because I and lots of other people do both: you skim the TOCs of journals that interest you, looking for interesting-seeming titles. Conversely, I’ve never heard of anyone *just* looking at paper titles, without any other “prefilter” (how would you even do that?) And I don’t know of anyone who *just* reads everything that’s published in journal X, regardless of the title or any other feature of the paper. (Well, except Brad Anholt, but he’s an unusual case. He keeps up with the literature by reading *just* the introductions of every paper in several leading EEB journals.)
My larger puzzlement is with the notion that, if we somehow got rid of journals, that our ability to filter and evaluate both individual papers and the scientists who write them would somehow be improved or “fairer” or more “objective” or something. Or maybe the idea is that, if we got rid of journals (at least the selective ones), we would, and should, stop evaluating science on any criteria other than “technical soundness” and “whether I personally happen to like it”.
Now seems like a good time to re-up some old posts on this:
https://dynamicecology.wordpress.com/2012/11/25/how-random-are-referee-decisions/
https://dynamicecology.wordpress.com/2012/07/16/citation-concentration-filtering-incentives-and-green-beards/
https://dynamicecology.wordpress.com/2013/04/08/selective-journals-vs-social-networks-alternative-ways-of-filtering-the-literature-or-po-tay-to-po-tah-to/
LikeLiked by 1 person
“I’ve never heard of anyone *just* looking at paper titles, without any other “prefilter” (how would you even do that?)”
The way that lots of people do it. By setting up keyword notifications from Google Scholar, PubMed, etc.
LikeLike
Yes, sorry should’ve been clearer on what I meant. I was mentally including “keyword searches” as a kind of “prefilter”.
LikeLike
Of course, once upon a time at least, the choice of a database was itself a prefilter. Fly-by-night predatory journals weren’t in Web of Science (still aren’t??), PubMed was (still is?) strong in some fields and not in others. To some extent, Google Scholar’s strength (inclusiveness) is also its weakness (no help in prefiltering).
LikeLike
Thanks for the post, Stephen.
I do, of course, still list journal names in resume, grant applications, etc., — basically wherever people are likely to judge a paper by its cover rather than by its content.
Let me tell an anecdote — someone listed a paper in its resume with the journal acronym. Someone else commented “What is this fake Indian journal?”, to which I replied that it was actually [very well regarded journal in the field] — the immediate reply was “Well this changes everything, the guy looks good”.
On the other hand, the issue I have with “journals are a reflection of the type of work” is that it is not really true! Journals are a reflection of where your paper was not rejected from. We usually start with a target journal, and then move down the food chain until someone says yes. Sometimes up. But we are not journalists pitching this to publications. And journals don’t have an editorial line. The idea of a “culture” at most journals is dubious.
LikeLiked by 1 person
Thanks for replying, Timothée! As is often the case, I’d bet we actually agree on a fair bit of this, but clearly there are some bits of interesting disagreement. I would actually take your anecdote as a piece of evidence in favour of my view that journal names help by honestly signaling about the paper. When the journal name was (accidentally) concealed, the person in your anecdote made a bad inference; when it was revealed, they updated and likely improved it. (Whether one can reason safely this way about a SINGLE paper is of course a statistical issue; as you saw, I really focused more on inferences about SETS of papers.)
One point you make that I don’t know whether to agree with or not is your closing. You say “The idea of a culture at most journals is dubious”. I just don’t know. I know lots of journals that clearly DO have cultures (Am Nat, Ecology, Evolution, Evol Ecol Research, Oikos, just off the top of my head). And I can think of some that don’t: (PLOS, obviously, and its ilk, and the new FACETS). And by definition, I guess, the preprint servers lack “culture”. But where is the “most journals”? I don’t know. There are a lot of journals I’m not familiar with, and I might suspect they lack a “culture” but it may only be that I don’t know what it is!
LikeLike
Saying which journal your paper is published in? It emphasizes the kind of paper you have, and what audience it is designed for. A paper in an entomology journal is one that you think entomologists will want to read. A paper in an behavioral ecology journal is one that behavioral ecologists will want to read. A paper in Am Nat is one that straddles a few lines and people who think integratively will want to read. Our smaller communities, as long as they’re not built on exclusivity, are what builds progress.
(On the other hand, making a big whoop about the impact factor of a journal is somewhere between annoying and counterproductive to the endeavor of science.)
LikeLiked by 3 people
+1 the journal should be about target audience and receiving feedback from a particular community. This is useful information (I do think the idea of overlay journals for preprints could help here – ie provide a stamp of recommendation from trusted sources).
Impact factors etc are just an indirect measure, typically less helpful than knowing other interesting/useful work (wrt your interests) has been published there before. For example a colleague is currently targeting a manuscript according to journals ranked by impact factor. I would argue that their work would ultimately be better served and be more likely to be cited if target audience was the priority over impact factor.
During my PhD I had a paper go through a series of reviews at a very high impact factor/prestigious journal. Ultimately it was rejected as not quite exciting enough. Rather than aim for the next tier or other high impact factor journal I just submitted it to a well-regarded but much lower impact journal because I thought the target audience was more appropriate. I got very helpful feedback and it was published without too much nonsense involved. Perhaps I should have ‘aimed higher’ but I think it was the appropriate choice and have no regrets.
LikeLike
RE: overlay journals, in case people don’t know what I’m referring to see here: https://gowers.wordpress.com/2015/09/10/discrete-analysis-an-arxiv-overlay-journal/
I think this helps solve a number of problems and is a good compromise between the ‘no journals’ and ‘business as usual’ folk.
LikeLike
See also: https://plus.google.com/+TimothyGowers0/posts/VcRfMETfSkd
LikeLike
I agree with your arguments here. I find a journal name can give some additional information about the scope a paper is likely to have, the level of specialisation of audience it’s intended for and (sadly) an instant answer to whether I’ll be able to get a hold of the paper easily or not (bless you, open source journals…).
LikeLike
A couple of thoughts:
1) Simply going down a list of impact factors is frankly bonkers because that’s just trying to send your orange to places that only publish apples. What a waste of time and energy! It’s due diligence to find out where oranges belong. It’s the primary reason papers get rejected without review. It might be a great orange, but it will never be an apple.
2) In a world of different journal types, including “big buckets” there will have to be multiple ways of sorting. But what does that sorting consist of when journal name doesn’t help? In the booth at ESA one year I asked some students how they found what to read–because they told me they never think about journal names and don’t really know one from the other. They said, they look for an author’s name they know or a famous affiliation. That’s really not good.
3) It’s not good because great work that’s innovative often comes from people with fresh ideas. When we started the Student Paper award, people scoffed but it turns out that 1/3 or more papers qualify. We made it so people without institutional funds could publish at no cost to encourage postdocs etc. The Natural History section gets submissions from all kinds of people-scientists at the Florida Water Reclamation District or Colombian researchers who spend all their time in the Amazon lowland (and didn’t realize their paper had gone viral). The journal name helps them get read (well, if anyone is paying attention to journals as niches of course)
The editorial board works hard to help people with promising ideas or observations revise their work so that it’s clearer and stronger and the innovation reaches the reader. We work hard in the office to help people not used to thinking about presentation get their figures up to excellent quality and the editorial team works hard on the manuscript editing and typesetting. We all view it as a service to the community. So, yeah, old school journals with distinct niches can have hoops, but we hope each article has some of the experience described in this blog post: https://dynamicecology.wordpress.com/2012/10/16/the-paper-that-ecology-rejected-that-later-won-the-mercer-award/
In other words, while I think it’s great that there are lots of alternatives are out there serving lots of needs, I do worry that big buckets without hoops don’t serve everyone equally.
LikeLike
Thanks, Trish, and thanks for bringing up the issue of access to reader attention for new folks. This is a major service journals do, I think, that preprint servers and ‘big box’ journals can’t: we even out the playing field of privilege-because-famous that you describe. I shoudl have linked to this earlier post about that: https://scientistseessquirrel.wordpress.com/2015/12/14/post-publication-peer-review-and-the-problem-of-privilege/
LikeLike
Been thinking about this all day…what you’ve given as your reasons for your strategy in following certain journals makes sense…but what is an ideal strategy for a nonprofessional interested broadly in science, with particular interests in fields that are scattered, who is not as focused as you are and has no compelling reason to be?
We take Nature and Science because they are generalist journals; we read in them what interests us first, and then what we have time for. In the same vein we take several medical journals (JAMA, New England Journal of Medicine.) As we live in a small town, away from any academic library, and cannot afford all the journals that look interesting (and might take me deeper into my particular interests), this seemed a good strategy decades ago when we committed to making at least weak attempts to “keep current.” But your post suggests that another approach might be more useful now, when I’m aware that my interest in physics (for instance) has diminished as I’ve fallen farther behind. Maybe it’s time to choose differently?
Not sure. Not expecting clear direction, either. But if you have thoughts on how people not working professionally in a field–but very interested in it–might best use their journal subscription resources, I’d be glad to hear them.
LikeLike
This is a really great question, and (as you often do) you’ve turned me to look in a slightly different direction. I don’t have a good answer; I know best (and think almost entirely about) our literature and its function as an “in house” thing. For someone who is curious but has little background, I’d recommend following some of the best science journalists and bloggers (Zimmer, Yong, etc; the NY Times Science section) – and that can help anyone, but it alone won’t satisfy the kind of person who subscribes to a few journals!
I can’t think of a better pair of journals for the broadly interested than Nature and Science (despite my occasional swipes at them). Their coverage is not even across fields, of course; there are fields in which much of the best stuff is there, and fields in which little of it is. It helps that they cover other things in news-style bits. However, as the literature has continued to grow, Nature and Science have inevitably faded in any presumed role of “read these and you’ll be generally up-to-date”. I don’t know of a strategy for doing this now, and I’m skeptical that it’s even possible.
I can’t think of a better approach than “Nature + Science + science journalism with followup”. By “followup” I mean finding the paper(s) that spurred the story, so using the journalism as a discovery device, not (just) as a source of content. A complementary strategy is to set up Google Scholar alerts, but that works best for very focused discovery. One important point to make, though, is that in terms of reading papers you do not need to subscribe to a ton of journals. You might or might not have access to some via your local library (this varies tremendously) or via a friend or colleague who has access to a university library. Also, simply emailing the corresponding author and asking for a PDF should work a large fraction of the time. (It would be an interesting sociological experiment to try this three ways and to compare return rates: A. not identifying yourself in particular; B. identifying yourself as an interested layperson with some background; C. identifying yourself as an SF author. I honestly don’t know which would get the highest return! It will never be 100% but I bet it will be pretty good).
I hope others will have good ideas and will chime in.
LikeLike
Not that Am Nat is necessarily what you are interested in but as a nonacademic, you might be interested in our lay summary page. Not every author writes one up, but many of them do an excellent job of explaining what their papers are about and why that’s interesting. http://www.amnat.org/an/newpapers.html
LikeLike
Great post Stephen and your comment about Nature is spot on 🙂 Totally agree with you that journals have ‘cultures’ and that as author and reader you know what to expect from the journal title
LikeLike
This is a great post, on a very important topic. Very convincing that scope matters, and there is still an important place for journal brand in science publishing.
I think it is also relevant here to discuss the importance of selecting the journal brands you would like to support (either by publishing or reviewing) not only based on the scope/impact combination,
but also based on publishing practices and uses of fees. Perhaps there is a place for some sort of ‘Ethical Publishing Certified’ label for journals.
Jeremy Fox touches on the choice for society journals:
https://dynamicecology.wordpress.com/2013/01/24/advice-how-to-decide-where-to-submit-your-paper/
Some additional reading here:
http://www.cbc.ca/news/technology/academic-publishers-reap-huge-profits-as-libraries-go-broke-1.3111535
and here:
http://www.theatlantic.com/science/archive/2016/01/elsevier-academic-publishing-petition/427059/
LikeLike
I think that’s best left to individuals to judge according to their own values. Trying to come up with some kind of “certified ethical” label just amounts to some people trying to impose their values on others. And the trouble is, whoever feels like putting in the effort to invent and promote a labeling scheme is probably going to be a passionate advocate who thinks their own views are Right and everyone else’s are Wrong. For instance, some very prominent open access evangelists are very critical of the value of most scientific society activities and how scientific societies spend the money they earn from journal subscriptions. I respectfully disagree, and would not want to see “anything that seems to inhibit universal adoption of author-pays open access is unethical” adopted as as an “ethical” principle in scientific publishing. This is just one example off the top of my head; the broader point is that “certified ethical” labeling schemes aren’t appropriate when there’s widespread reasonable disagreement about what’s ethical.
LikeLiked by 3 people
Excellent point, thanks for your reply. For the record, I was thinking more about relating corporate publishing profit margins to article processing charges etc., rather than specifically about mandatory OA, but I do understand that there is still considerable debate on what is ethical. Hopefully people do consider carefully for themselves what is the best journal for them to publish in, not only which has the highest IF within their field.
LikeLike
Pingback: How many non-peer-reviewed publications should a scientist produce? | Jeff Ollerton's Biodiversity Blog
Referred to this in my recent post: https://jeffollerton.wordpress.com/2016/08/31/how-many-non-peer-reviewed-publications-should-a-scientist-produce/
LikeLike
Pingback: Friday links: Lego grad student, ESA awards, and more | Dynamic Ecology
Pingback: It isn’t easy being an applied ecologist – working on crops limits publication venues | Don't Forget the Roundabouts
Pingback: How do you choose a journal when it’s time to submit a paper? | Scientist Sees Squirrel