Aspiring to greater intellectual humility in science
Rink Hoekstra & Simine Vazire, 2021
"We provide a set of recommendations on how to increase intellectual humility in research articles and highlight the central role peer reviewers can play in incentivizing authors to foreground the flaws and uncertainty in their work, thus enabling full and transparent evaluation of the validity of research."
"When reviewers push for authors to own their limitations, this is not only good for science but may even be in line with (some) authors’ ideals. Of course, reviewers can also use positive comments to reshape incentives; that is, when authors go the extra mile to put the limitations of their work front and centre, reviewers can point this out."
0.1. The abstract should describe the limitations of the study and boundary conditions of the conclusion(s).
0.2. Titles should not state or imply stronger claims than are justified (for example, causal claims without strong evidence).
Introduction
1.1. The novelty of research should not be exaggerated.
1.2. Selective citation should not be used to create a false sense of consistency or conflict in the literature."
2.1. The methods section should provide all the details that a reader would need to evaluate the soundness of the methods and to conduct a direct replication.
2.2. The timing of decisions about data collection, transformations, exclusions and analyses should be documented and shared."
3.1. Detailed information about the data and results (including informative plots and information about uncertainty) should be provided.
3.2. It should be transparent which analyses were planned and where those plans were documented; weaker conclusions should be drawn to the extent that analyses were susceptible to data-dependent decision-making."
3.3. Inferential statistics should not be used in a way that exaggerates the certainty of the findings; alternatives to dichotomous tests should be considered."
5.1. Insist that press releases and reporters capture the limitations of the work, and correct outlets that exaggerate or misrepresent.
5.2. Encourage criticism, correction and replication of the work, and respond non-defensively when errors or contradictory evidence are brought to light.
5.3. When appropriate, retract papers, issue corrections or publish ‘loss of confidence’ statements."
Add comment