Logical Fallacies and the Misuse of RCT Findings: Lessons from Diagnostic Statistics

This chapter offers an excursion through many of the important analogies encountered on comparing the conceptual frameworks for diagnostic statistics and null hypothesis significance testing (NHST). We shall meet three common logical fallacies that can prove influential in overestimating the probability of an intervention being effective, given a statistically significant outcome in favour of the intervention. In keeping with previous work, this conditional probability will be referred to as the positive predictive value (PPV) of the hypothesis test for the given study hypothesis. Relatedly, we shall see how researchers can be duped into a false reliance on statistical significance from a single randomised controlled trial (RCT) as an indicator of the reputability of the study hypothesis, while overestimating, and neglecting the importance of, the PPV. Recognition of the need for a higher PPV points to the fallibility of the RCT, in keeping with other study designs, as a source of scientific truth. In addition to training students to recognise and circumvent common misapplications of conditional probabilities in diagnostic statistics and hypothesis testing, the content of this chapter warns against some misconceptions in traditional representations of the relationships between prevalence and diagnostic statistics in medical textbooks and related expository papers. While this ought to support modern teaching on the theme of diagnostic testing, the aim in view is that of preparing the way for consideration in Chap. 4 of the parameterisation and management of the PPV using Bayes’ rule specifically within the context of NHST.