The publishing sector has a problem. Scientists are overwhelmed, editors are overworked, special issue invitations are constant, research paper mills, article retractions, journal delistings… JUST WHAT IS GOING ON!?
@MarkHanson@paolocrosetto I know so many academics who agree that there are too many publications, but every single one of their publications is still ground-breaking 🤷♂️
First, things first: growth in articles published each year has outpaced the scientists doing the publishing. With #PublishOrPerish, we all face an ever-increasing workload (writing, reviewing, editing…). It’s been rough.
Strain itself is neutral: this could be a welcome change! Are we becoming more efficient? Are we combatting biases (academic racism, positive result bias)?
If that’s all it were, the solution to strain would be to build a better infrastructure.
We see that certain groups are major drivers of this article growth, in some cases seemingly out of nothingness. This includes your classic publishers like #Elsevier and #Springer, but also the upstarts #Frontiers and… most significantly #MDPI.
In numbers, there were nearly 1 million more articles per year published in 2022 (2.8m) compared to 2016 (1.9m). MDPI takes the lion’s share at 27% of that growth, with Elsevier (16%) a distant 2nd.
I could be nuanced (it's in the preprint!). But let’s be frank: it’s special issues.
“Dear Dr ___, your preeminent work in [FIELDYOUDONTWORKIN] drew our attention to your [COPYPASTEPAPERTITLE] and we were thoroughly aroused. We invite you to submit to special issue with us, who love your preeminence. Yours faithfully, [NAMEWHOISAROUSED].”
The figure speaks for itself. With my leftover characters, instead I wanna ask y’all to send me screenshots of your favourite SI invitations! Hit me! 😀 4/n
So still… is it worth it? Strain itself is neutral. Maybe these special issues are just giving a voice to authors with less privilege?
Or maybe not. The publishers hosting special issues drastically reduced their turnaround times (TATs: submission to acceptance) - and let’s be clear, that’s INCLUDING revisions. 5/n
Now, it’s not our place to judge what an average TAT is supposed to be, but we’re very confident it’s not 37 days across all research fields. Experiment requests in fruit flies take weeks, whereas mice will take months.
TATs are also supposed to vary from article to article: some articles are great on 1st draft, some need a little TLC, and some need… a lot… Yet #MDPI journals in particular, across the board, accept everything in a blistering 37 days with almost no variation. 6/n
But it’s not just #MDPI: #Frontiers and #Hindawi also grew their share of special issues. One might argue: “These are just labels publishers use. The peer review process is the same.”
Au contraire mon ami : no it’s not. Special issues have lower TATs. They’re intended to be lax. They’re for authors to voice ideas that could turn out to be wrong, but advance the conversation in the field. That’s what they used to be at least… and what made them “special.” But I digress… 7/n
We also looked at rejection rates (RRs), with some caveats: we took a publisher’s word at what their RRs were, and don’t know underlying methods. But we figured RRs will at least be calc’d consistently within groups. We compared relative RRs over time and RRs compared to proportions of special issues.
Again, #MDPI was the maverick, with a unique decline in RRs over time. Not only that, but in both #Hindawi & MDPI, more special issues means lower RRs. The review process is not the same. 8/n
Lastly let’s talk #ImpactFactor (IF). Reminder: IF = avg cites/doc articles in a journal receive within 1st 2y. IF values total cites.
IFs are going up 📈: they’re literally being inflated like a currency. So if you see a journal celebrating its year-over-year increase in IF, you’ve gotta normalize for inflation. This inflation accompanies the huge crush of special issues from earlier. But(!) a citation network-adjusted rank (Scimago Journal Rank, SJR) hasn’t changed accordingly. What gives? 9/n
Well, SJR is complex, but the main thing is it doesn’t reward self-citations, or circular citations from so-called “citation cartels.”
In other words:
** IF just cares about total citations, but doesn’t pay attention to where they come from.
** SJR pays attention, and doesn’t reward you or your buddies for disproportionately citing each other.
We developed a new metric that we call “Impact Inflation.” Impact Inflation is the ratio of Impact Factor to Scimago Journal Rank (IF/SJR). Because IF values total cites (no matter the source), but SJR fails to reward authors aggressively self/co-citing, IF can become extremely inflated compared to SJR for journals hosting citation cartels.
Key point: Impact Inflation is a metric that shows to what extent a journal has succumbed to Goodhart’s law. And well… once again #MDPI leads the pack. 12/n
Talking within-journal self-cites, once again #MDPI has the highest rates
What’s more we also see groups like #Hindawi have higher Impact Inflation, but normal self-cite levels. What gives?
Well, SJR also weights a citation based on where it comes from, and because MDPI journals aren’t well-cited (except by themselves), their citations aren’t worth much. And because MDPI growth came out of nowhere, they’re now exporting huge numbers of citations to others, including a penchant for Hindawi 13/n
So where does that leave us? Well, it’s easy to talk about #MDPI because… scroll up. But fundamentally we need to address strain. We’re all overworked, and we can’t let this go on.
Our metrics tell us this growth isn’t rigorous science. Special issues are lowering standards, which nets groups like MDPI more articles, and more money 💱. We don’t have revenue data, but for-profit gold OA ties revenues to articles published. So it’s no surprise that some groups are gonna spam engines of growth 14/n
Science needs accountability. The public needs to trust “peer-reviewed” papers have some minimum standard. These crazy-prolific special issues are damaging the authority and integrity of science.
It’s also costly: millions of scientists writing, reviewing, editing, and for what? These extra ~1m annual articles aren’t necessary. What’s more: we’re under-describing the strain because we’re only using journals indexed in both Scopus and Web of Science. Surprise! It’s actually even worse 🙃 15/n
That said: we’re just four white guys who all got fascinated with the craziness of the publishing sector. But you, the reader, can help. Publishing scientific articles can’t be like ordering fast food: “I’d like one special issue article please, hold the critiques.”
Special issues need to be a rare treat. A “sometimes” food. And when you’re invited to publish in one, or host one, that invite shouldn’t come from an algorithm. We should try to establish this basic #ResearchCulture 16/n
We need policies that treat special issues differently because they are. We need guidelines from #COPE on a reasonable minimum rigour for #peerreview. We need standard reporting of key metrics like RRs, profit margins, etc… We need leadership, and thank you for all you’ve already done and all you’re going to do. We’re up to chat! 17/n
Now the mushy stuff: Pablo, @paolocrosetto Dan, it’s been an incredible privilege to work with you all on this. I learned a ton through this project on coding and reproducible research practices.
Also: thanks for putting up with me, I know I’m a lot. As we’ve heard many times from folks over the last months: “this work you guys are doing is really important.” I believe it. Still banned from the text though. 18/n
A last point: we really hummed and hawed about if and how we could release scripts and data, but we just can’t right now. Lawyers told us not to. We’re like… 99% sure we didn’t do anything risqué, but these things can’t be rushed. We’ll update the preprint if/once we’ve confirmed everything. Sorry about that, but hope it’s understandable. 19/n
That’s it. If you’ve made it this far, thanks for reading. I hope this work can help you start some much-needed conversations at your local level.
If you want to chat, I’ll personally be happy to try and carve out some time to. If so, best to send me an email. Let’s work together to be the change we want to see in #Science
20/20 - end
@MarkHanson This was a great read! However, I would be very careful about this chart in particular: using double y axes to show an association can be very fraught, and especially the "strain" annotation here sets off all kinds of alarm bells in my data viz brain. Here's some more reading on the topic: http://daydreamingnumbers.com/blog/dual-axis-charts/
@janeadams for sure! The only idea here is to show that at same time article growth was ramping up, PhDs awarded was plateau'ing. In Fig1supp1 we present it as a single-line ratio using a couple datasets. #DataViz 1/2
@MarkHanson@dingemansemark@petersuber@ct_bergstrom ks for this very nice thread and put data and words on a feeling! I suggest looking toward @PeerCommunityIn and @PeerCommunityJournal for non-profit and alternative way of publishing 🙂 I would have been curious to see the ratio between article production and money gain from the publisher perspective.
@AudreyBras I'm a recommender for PCI infections! Indeed, having niggling doubts now about what we do with this preprint:
We thought we had a plan (send to big publisher for credibility/visibility needed), but the response has been awesome. I, personally, am wondering if we could in fact send it to a Diamond OA like PCI without handicapping our ultimate goal of visibility/conversation-driving. The message it would send, but the potential audience we'd lose, it's such a weird dance right now.
@MarkHanson
Appreciate the work here, but not sure the prescriptions follow. Special editions are just one manifestation of a general pattern in for profit publishing - the obvious incentive to push as many through for as much of a processing charge as the prestige of the brand can bear. Why treat a symptom as a target for intervention? And why abandon the idea of infrastructure building? That seems to to take the underlying landscape as fixed, out of our control, which lends itself to the perennial appeal to funders, which to me is an abdication of our responsibility, agency, and a misattribution of the root causes and power dynamics in play.
One could see the same results and conclude faculty need militant unions to push back against the evaluation systems that make these metrics meaningful, or to organize alternative means of publication. Bibliometrics as you note are intrinsically gamed, but treating the very structure of bibliometrics (citations in traditional journal articles) as inevitable seems to do little to challenge that game, even with a new derived metric that metricizes how gamed one metric is vs another.
@jonny@MarkHanson To add to Jonny's comment, this is good empirical work but more needs to be done to situate the recent shifts, both historically & within the larger institutional dynamics of science. Publishers reflect/refract the incentive structures of their customers (both authors & institutions), which means publishers aren't best analyzed as the only entities with agency.
@jonny perhaps I'll respond by saying: read the paper! The Mastodon thread was written for a public excitement, but we've got the nuances in the proper paper.
Also SIs are the plurality of strain by a large margin, and our data suggest they are treated differently (across publishers no less, even the 'classic' ones), so the model of "as many articles as possible through SIs" is just not sustainable, nor expected to produce articles similar to standard issues.
@MarkHanson interesting thread. Is it reasonable to say that peer-based publish or perish pressures (jobs, grants, annual reviews, etc.) are the cause and the predatory journal practices are the effect (they simply feed because there’s something to feed on)?
Indeed, there's a flagrant conflict of interest: each paper accepted pays APCs, each paper rejected costs time and money to handle and pays nothing.
For journals to be honest, they should charge for submission to acknowledge the processing cost – presumably a smaller sum than current APCs – or not charge at all: decouple income from accepting papers.
@albertcardona@MarkHanson that would be even worse! It would just reverse the incentive not get rid of it, and lead to even more wasted time from scientists (an externality as far as publishers economics are concerned).
If they said, here is a submission fee, but if accepted there's no acceptance fee, would it have been received the same way?
The one issue with submission fees is that it would reduce the number of submissions, which at the limit would match that of acceptances. I doubt this would happen though, particularly for glamour journals were submissions can feel like buying a lottery ticket.
Thank you, that was a very interesting read. What an accurate picture of the Society for Neuroscience.
Having paid the membership from my pocket for 2-3 years after having attended SfN once, I realized it was a racket. Serving its own interests and not at all mine or those of any practicing neuroscientists, at least not the junior ones and much less if outside the US.
Thanks, and interesting thought. Haven't yet heard about the link between the society and JFRC. As far as I know the pivot from malaria research to neuroscience before opening its doors had to do with the Gates Foundation announcing their focus on neglected tropical diseases at the time of JFRC's preparatory meetings to choose the focus of the whole institute, and the influence of early advisors to the directorship such as Brenner and Borst.
@schoppik@MarkHanson@deevybee@albertcardona I find it hard to have many positive feelings about the society for neuroscience. As a theorist, I'm treated as a second class citizen (common with most neuroscience orgs though). As a European, I feel like I'm paying for an org that mostly lobbies for US-based neuroscientists, which feels unfair. In general, it seems to be an org that supports the model of gating access to participation in science behind the ability to pay, which feels like exactly the opposite of what we should be doing as a scientific community.
@neuralreckoning
I went to the thing for my first time this year. It was cool for lots of people to be there, but that was basically it - I wandered around trying to see stuff but gave up and just spent most of my time trying to see friends I hadn't seen in awhile/had only ever met online. Folks came around to my poster but mostly those that I already knew or were already on board. Hard to justify the like $2000 it cost to go all told. Doubt ill go again until I have something I like want to put in ppls hands and can get in a booth or smth.
Too big. Felt weird. Dont like the politics of "one big corporate nonprofit." Did manage to see a vendor zapping their own brain with a TMS machine which was a highlight.
@jonny I don't know if it's a consolation or not but your $400 registration fee probably more than covered an hour's salary for the CEO. Probably not two hours though.
@neuralreckoning@albertcardona@MarkHanson I still think science funders are the only people who can stop this mess. If HHMI or Wellcome or ERC said "you can have this grant money on one condition: you publish all resulting research on our OA platform".
They are the only player in this mess who can't claim "it's not my fault, the system made me do it"
Add comment