Edinburgh Research Explorer

Have standard formulas correcting correlations for range restriction been adequately tested? Minor sampling distribution quirks distort them

Research output: Contribution to journalArticle

Related Edinburgh Organisations

Open Access permissions

Open

Documents

  • Download as Adobe PDF

    Rights statement: The final version of this paper has been published in Educational and Psychological Measurement, All rights reserved. ©The Authors, 2017. It is available at: http://journals.sagepub.com/doi/full/10.1177/0013164417736092

    Accepted author manuscript, 788 KB, PDF-document

Original languageEnglish
Pages (from-to)1-35
JournalEducational and Psychological Measurement
Early online date26 Oct 2017
DOIs
Publication statusE-pub ahead of print - 26 Oct 2017

Abstract

Most study samples show less variability in key variables than do their source populations due most often to indirect selection into study participation associated with a wide range of personal and circumstantial characteristics. Formulas exist to correct the distortions of population-level correlations created. Formula accuracy has been tested using simulated normally-distributed data, but empirical data are rarely available for testing. We did so in a rare dataset in which it was possible: the 6-Day Sample, a representative subsample of 1208 from the Scottish Mental Survey 1947 of cognitive ability in 1936-born Scottish schoolchildren (70,805). 6-Day Sample participants completed a follow-up assessment in childhood and were re-recruited for study at age 77. We compared full 6-Day Sample correlations of early-life variables with those of the range-restricted correlations in the later-participating subsample, before and after adjustment for direct and indirect range restriction. Results differed, especially for two highly correlated cognitive tests; neither reproduced full-sample correlations well due to small deviations from normal distribution in skew and kurtosis. Maximum likelihood estimates did little better. To assess these results’ typicality, we simulated sample selection and made similar comparisons using the 42 cognitive ability tests administered to the Minnesota Study of Twins Reared Apart, with very similar results. We discuss problems in developing further adjustments to offset range-restriction distortions and possible approaches to solutions.

    Research areas

  • sStudy participation, range restriction, skew, adjustment formulas, distortion, statistical bias

Download statistics

No data available

ID: 43569903