Nearly two-thirds of all psychology research should be distrusted because it cannot be replicated, according to a troubling paper in Science.
This study reports work by the Reproducibility Project, an initiative at the University of Virginia in Charlottesville which was launched in 2011 after a succession of scandals in psychology research. Brian Nosek and a huge team of co-authors tried to reproduce the results of 100 articles in three leading journals. They found that only 39 could be replicated.
Although it is impossible to say whether the conclusions of a paper are true or false only because its results cannot be reproduced, it does suggest that it might not stand up to scrutiny. An expert in medical statistics, John Ioannidis, of Stanford University, told Nature that he fears that the true level of non-reproducible research may be as high as 80%.
Reproducibility is at the heart of the scientific method. As Nosek writes in Science, “Scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence. Even research of exemplary quality may have irreproducible empirical findings because of random or systematic error.”
Part of the problem is that promotion and prestige depend mainly on innovation and new research; there are few kudos for confirming another scientist’s results. Nonetheless reproducibility is vital: “Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not.”
“We see this is a call to action, both to the research community to do more replication, and to funders and journals to address the dysfunctional incentives,” Nosek told the New York Times. He says that research in other disciplines needs to be evaluated as well.
This article is published by
and BioEdge under a Creative Commons licence. You may republish it or translate it free of charge with attribution for non-commercial purposes following these guidelines
. If you teach at a university we ask that your department make a donation. Commercial media must contact us
for permission and fees. Some articles on this site are published under different terms.