Summary:

In medical education settings, among others, assessment scenarios can arise where an examiner with prior knowledge of examinee performance in one dimension is required to rate the same examinee(s) over a different dimension. Concerns ought to arise in these settings concerning examiner bias and as a means of enhancing assessment quality, these concerns ought to be addressed by a statistical test for such bias which suits the assessment design.

Article:

In medical education settings, among others, assessment scenarios can arise where an examiner with prior knowledge of examinee performance in one dimension is required to rate the same examinee(s) over a different dimension. Concerns ought to arise in these settings concerning examiner bias and as a means of enhancing assessment quality, these concerns ought to be addressed by a statistical test for such bias which suits the assessment design. The statistical test should also tolerate consistency in examinee performance across different dimensions which reflects examinee natural abilities, whether good or bad. In the paper Halos and horns in the assessment of undergraduate medical students: a consistency based approach, MacDougall et al. present new methodology with the above needs in mind and illustrate its use within the context of marking Year 4 Student Selected Components within the undergraduate medical curriculum at the University of Edinburgh.

Support links: