Have standard formulas correcting correlations for range restriction been adequately tested? Minor sampling distribution quirks distort them

Wendy Johnson, Ian Deary, Thomas J. Bouchard

Research output: Contribution to journalArticlepeer-review

Abstract

Most study samples show less variability in key variables than do their source populations due most often to indirect selection into study participation associated with a wide range of personal and circumstantial characteristics. Formulas exist to correct the distortions of population-level correlations created. Formula accuracy has been tested using simulated normally-distributed data, but empirical data are rarely available for testing. We did so in a rare dataset in which it was possible: the 6-Day Sample, a representative subsample of 1208 from the Scottish Mental Survey 1947 of cognitive ability in 1936-born Scottish schoolchildren (70,805). 6-Day Sample participants completed a follow-up assessment in childhood and were re-recruited for study at age 77. We compared full 6-Day Sample correlations of early-life variables with those of the range-restricted correlations in the later-participating subsample, before and after adjustment for direct and indirect range restriction. Results differed, especially for two highly correlated cognitive tests; neither reproduced full-sample correlations well due to small deviations from normal distribution in skew and kurtosis. Maximum likelihood estimates did little better. To assess these results’ typicality, we simulated sample selection and made similar comparisons using the 42 cognitive ability tests administered to the Minnesota Study of Twins Reared Apart, with very similar results. We discuss problems in developing further adjustments to offset range-restriction distortions and possible approaches to solutions.
Original languageEnglish
Pages (from-to)1-35
JournalEducational and Psychological Measurement
Early online date26 Oct 2017
DOIs
Publication statusE-pub ahead of print - 26 Oct 2017

Keywords

  • sStudy participation
  • range restriction
  • skew
  • adjustment formulas
  • distortion
  • statistical bias

Fingerprint

Dive into the research topics of 'Have standard formulas correcting correlations for range restriction been adequately tested? Minor sampling distribution quirks distort them'. Together they form a unique fingerprint.

Cite this