Assessing the reliability of the borderline regression method as a standard setting procedure for objective structured clinical examination

• Main Authors: Sara Mortaz Hejri, Mohammad Jalili, Arno M. M. Muijtjens, Cees P. M. Van Der Vleuten
• Format: Article
• Language: English
• Published: Wolters Kluwer Medknow Publications 2013-01-01
• Series: Journal of Research in Medical Sciences
• Subjects: Borderline regression method; objective structured clinical examination; reliability; standard setting
• Online Access: http://www.jmsjournal.net/article.asp?issn=1735-1995;year=2013;volume=18;issue=10;spage=887;epage=891;aulast=Hejri
• ISSN: 1735-1995; 1735-7136

Background: One of the methods used for standard setting is the borderline regression method (BRM). This study aims to assess the reliability of BRM when the pass-fail standard in an objective structured clinical examination (OSCE) was calculated by averaging the BRM standards obtained for each station separately. Materials and Methods: In nine stations of the OSCE with direct observation the examiners gave each student a checklist score and a global score. Using a linear regression model for each station, we calculated the checklist score cut-off on the regression equation for the global scale cut-off set at 2. The OSCE pass-fail standard was defined as the average of all station′s standard. To determine the reliability, the root mean square error (RMSE) was calculated. The R2 coefficient and the inter-grade discrimination were calculated to assess the quality of OSCE. Results: The mean total test score was 60.78. The OSCE pass-fail standard and its RMSE were 47.37 and 0.55, respectively. The R2 coefficients ranged from 0.44 to 0.79. The inter-grade discrimination score varied greatly among stations. Conclusion: The RMSE of the standard was very small indicating that BRM is a reliable method of setting standard for OSCE, which has the advantage of providing data for quality assurance.