jonny,
@jonny@neuromatch.social avatar

Think about the difference between opening an issue to report a bug vs. Submitting some review on pubpeer about a critical flaw in a published paper.

One is welcome, at worst a "friend telling you about food stuck in your teeth" correction that you cant see but other people can. (Sure some people raise issues like assholes, but solving people being assholes isnt really in scope). The other is potentially career ending, always confrontational even if we want it to be constructive, the amount of times it has ended up well is a vanishingly small fraction of all criticisms like that.

Whats different? Papers are like software in that they always have some bugs somewhere, are embedded in a moving field that can shift underneath it, they make some claims on reality and either deliver or dont. They are different in that papers arent expected to be amended, often thats literally impossible. Review is "done" when a paper is published. Review happens at the "end" of a paper rather than throughout its life. You cant deprecate a paper - "this was good at the time, but we have updated our understanding." All the things that make it so someone suggesting something you could fix being an act of kindness and collaboration are forbidden by the traditional review and publishing. (A lot more subtlety but im longposting on a microblogging medium not writing an encyclopedia rn)

A recent example im thinking of is this issue I raised with @linkml that was the underlying cause of a decent number of hard to diagnose bugs:
https://github.com/linkml/linkml/issues/1839

Did that invalidate some claims made in the docs and published lit re: importing and extensibility much like errata in a paper? Yes! Was it a huge deal, fatal to the reputation of the project? Absolutely not!!! In fact a noisy bug tracker is a sign of HEALTH. there isnt a reply there because I discussed it in person with the devs later, and theres a relatively obvious fix to be made, but I meant it in cooperation, as a compliment - I care enough about your work to investigate why it isnt working - and it was received as such. Im working on a patch now. Colleagues working together to make something work!

If youve ever criticized a paper before, you know that simply is not the way it goes. I have a lot of people who I love dearly who do a lot of work I think is fatally flawed, but I wouldn't dare criticize them because it could put their career in danger. They arent bad or stupid people, but their lab does things in a particular way, etc. And so everyone loses out: the work is worse, the work is stuck in time and becomes irrelevant within discipline and misleading outside of it, the labs never improve, the working conditions never improve, we create protective silos around ourselves to survive.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • random
  • DreamBathrooms
  • mdbf
  • ethstaker
  • magazineikmin
  • GTA5RPClips
  • rosin
  • thenastyranch
  • Youngstown
  • InstantRegret
  • slotface
  • osvaldo12
  • kavyap
  • khanakhh
  • Durango
  • megavids
  • everett
  • cisconetworking
  • normalnudes
  • tester
  • ngwrru68w68
  • cubers
  • modclub
  • tacticalgear
  • provamag3
  • Leos
  • anitta
  • JUstTest
  • lostlight
  • All magazines