Professor Alexander Bird, Professor of Philosophy and Medicine, King's College London, gives a talk for the Centre for Evidenced Based Medicine.
The replication (replicability, reproducibility) crisis in clinical medicine and in other fields arises from the fact that many apparently well-confirmed experimental results are subsequently overturned by studies that aim to replicate the original study. The culprit is widely held to be poor science: questionable research practices, failure to publish negative results, bad incentives, and even fraud. In this paper I argue that the high rate of failed replications is consistent with high quality science. We would expect this outcome if the field of science in question produces a high proportion of false hypotheses prior to testing. If most of the hypotheses under test are false, then there will be many false hypotheses that are apparently supported by the outcomes of well conducted experiments and null-hypothesis significance tests with a type-I error rate (alpha) of 5\%. Failure to recognize this is to commit the fallacy of ignoring the base rate. I argue that this is a plausible diagnosis of the replication crisis and examine what lessons we thereby learn for the future conduct of medical science.
