Prospective meta-analysis: Stimulating progress in parapsychology

In 2012, the authors launched the Koestler Parapsychology Unit (KPU) Registry for Parapsychological Experiments (https://koestlerunit.wordpress.com/study-registry/ ). Study registration is a valuable tool that helps to eliminate or identify many questionable research practices in individual studies and thereby enhances the methodological quality of an area of research. Researchers employ meta-analysis to quantitatively combine methodologically similar studies. When conducting a meta-analysis, they must make decisions about what data to include and what statistical methods to use. If researchers do this after knowing the outcomes of the studies, the decisions may be biased. Pre-registration of individual studies does not eliminate potential biases that emerge from decisions during a meta-analysis. Retrospective meta-analyses are analogous to exploratory research because the analyses are often adapted to the data. Prospective meta-analyses are analogous to preregistered confirmatory research because the analyses and the data that will be included are specified before the results are known.The present paper uses ganzfeld ESP research to illustrate the limitations of retrospective meta-analysis, and to highlight how prospective meta-analysis can help to resolve debates over the evidence for psi and stimulate progress in parapsychology. For the first time, we present a summary of the KPU ganzfeld ESP studies, and note the decisions facing anyone seeking to evaluate their combined outcome. We then discuss the wider ganzfeld ESP database and associated debates about the methodological decisions for meta-analyses. The extensive discussions and debates about the Milton and Wiseman (1999) meta-analysis demonstrate the latitude in making decisions for a retrospective meta-analysis and, most importantly, the potential for bias either pro or con the psi hypothesis.We then introduce a registration-based prospective meta-analysis of ganzfeld ESP studies and describe the differences from the more common prospective meta-analysis in medical research. The ganzfeld prospective meta-analysis will be preregistered on the KPU Study Registry following review by parapsychologists and critics. In addition to preregistering the statistical methods, this meta-analysis will use future study registrations to prospectively decide which studies will be included. Any qualifications or modifications for the use of a study will also be specified prospectively on the list of included studies. This approach to meta-analysis does not limit process oriented research or innovation, but simply specifies what studies will be included in a subsequent meta-analysis before the results of the studies are known. The benefits of registration-based prospective meta-analysis as proposed here include (a) each study is preregistered and precludes most questionable research practices; (b) the meta-analysis does not require the increasingly complex methods that are being used to evaluate possible methodological biases; (c) the decision to include a study in the meta-analysis and decisions about possible qualifications for the use of study data are specified prospectively at the time the study is registered, which eliminates biases (pro or con) from methodological decisions after the study results are known and also allows adaptation to the unique characteristics of a study; (d) the properties of psi will be revealed without being obscured by methodological noise and biases; (e) potential critics of the meta-analysis outcome (pro or con) can be given the opportunity to comment on the meta-analysis plan; and (f) the field of parapsychology will be recognized to be a methodological leader.Finally, we consider how prospective meta-analysis could be applied to other lines of parapsychological research. By pioneering high standards of methodology and research synthesis, parapsychologists send an important message to researchers in other fields (Watt & Kennedy, 2015).