Abstract / Description of output
The authors introduce a consistency-based approach to detecting examiner bias.
On comparing intra-class correlation coefficients on transformed data for supervisor continuous performance and report marks (ICC1*) with those for supervisor continuous performance and second marker report marks (ICC2*), a highly significant difference was obtained for both the entire cohort (ICC1* = .72, ICC2* = .30, F = 2.47, p < .0005 (N = 1085)) and the subgroup with high supervisor ratings for continuous performance (ICC1* = .62, ICC2* = .24, F = 1.97, p < .0005 (n = 952)). A strong halo effect was detected and preliminary evidence was obtained for the presence of a strong horn effect for students with
lower scores, thus providing a basis for future research.
On comparing intra-class correlation coefficients on transformed data for supervisor continuous performance and report marks (ICC1*) with those for supervisor continuous performance and second marker report marks (ICC2*), a highly significant difference was obtained for both the entire cohort (ICC1* = .72, ICC2* = .30, F = 2.47, p < .0005 (N = 1085)) and the subgroup with high supervisor ratings for continuous performance (ICC1* = .62, ICC2* = .24, F = 1.97, p < .0005 (n = 952)). A strong halo effect was detected and preliminary evidence was obtained for the presence of a strong horn effect for students with
lower scores, thus providing a basis for future research.
| Original language | English |
|---|---|
| Pages (from-to) | 116 |
| Number of pages | 128 |
| Journal | Journal of Applied Quantative Methods |
| Volume | 3 |
| Issue number | 2 |
| Publication status | Unpublished - 2008 |