A Predictive and Construct Validity Study of a High-Stakes Objective Clinical Examination for Assessing the Clinical Competence of International Medical Graduates

BACKGROUND: The purpose of this study was to investigate the predictive and construct validity of a high-stakes objective structured clinical examination (OSCE) used to select candidates for a 3-month clinical rotation to assess practice-readiness status. SUMMARY: Analyses were undertaken to establish the reliability and validity of the OSCE. The generalizability coefficient (Ep(2)) for the assessment scores (checklist, global, and total) were all high, ranging from 0.73 to 0.84. Two discriminant analyses (promotion to the 3-month rotation and pass/fail status on the rotation) provided evidence of predictive validity with a 100% correct classification rate in the pass/fail rotation results. Factor analysis results provided evidence of construct validity with four factors identified: Clinical Skills, Internal Medicine, General Medical Knowledge, and Counseling. The known group differences between licensing status and residency experience also provided evidence of construct validity. CONCLUSIONS: The results are encouraging for the predictive and construct validity of the OSCE as an assessment of clinical competence.