Moving beyond the nuts and bolts of score reliability in medical education: Some valuable lessons from measurement theory

Research output: Contribution to journalArticlepeer-review

Abstract

In the light of recent debates within medical education concerning the shortcomings of the psychometric model, there is a need to prepare the way for better practise among the medical education community in the choice of reliability estimates for test scores. In this paper, therefore, I seek to achieve four main objectives: (1) to draw attention to the potential of Tarkkonen’s rho and Raykov’s reliability estimate to serve as alternative reliability estimates to Cronbach’s alpha, including when considering multidimensional test scores; (2) to highlight the range of measurement conditions and experimental designs under which Cronbach’s alpha is likely to show considerable negative or indeed, positive bias; (3) to explain the dangers of such bias and (4) to argue for the practise of testing compatibility of scores with measurement model properties prior to choosing reliability estimates so as to safeguard against such bias. In so doing, I shall highlight the need for strengthening collaborations between clinicians and measurement theorists or specialist statisticians in order to make such procedures more practicable and lend greater credibility to the psychometric model.
Original languageEnglish
Pages (from-to)643-664
Number of pages22
JournalAdvances and Applications in Statistical Sciences
Volume6
Issue number7
Publication statusPublished - Jan 2011

Fingerprint

Dive into the research topics of 'Moving beyond the nuts and bolts of score reliability in medical education: Some valuable lessons from measurement theory'. Together they form a unique fingerprint.

Cite this