Evaluating the validity evidence of an OSCE: results from a new medical school
Abstract Background To prevent the problems of traditional clinical evaluation, the “Objective Structured Clinical Examination (OSCE)” was presented by Harden as a more valid and reliable assessment instrument. However, an essential condition to guarantee a high-quality and effective OSCE is the ass...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
BMC
2018-12-01
|
Series: | BMC Medical Education |
Subjects: | |
Online Access: | http://link.springer.com/article/10.1186/s12909-018-1421-x |
id |
doaj-fb76f8d93deb41acb369ef0b6d402aff |
---|---|
record_format |
Article |
spelling |
doaj-fb76f8d93deb41acb369ef0b6d402aff2020-11-25T03:44:57ZengBMCBMC Medical Education1472-69202018-12-011811710.1186/s12909-018-1421-xEvaluating the validity evidence of an OSCE: results from a new medical schoolVanda Yazbeck Karam0Yoon Soo Park1Ara Tekian2Nazih Youssef3Lebanese American University-School of MedicineDepartment of Medical Education, University of IllinoisDepartment of Medical Education, University of IllinoisLebanese American University-School of MedicineAbstract Background To prevent the problems of traditional clinical evaluation, the “Objective Structured Clinical Examination (OSCE)” was presented by Harden as a more valid and reliable assessment instrument. However, an essential condition to guarantee a high-quality and effective OSCE is the assurance of evidence to support the validity of its scores. This study examines the psychometric properties of OSCE scores, with an emphasis on consequential and internal structure validity evidence. Methods Fifty-three first year medical students took part in a summative OSCE at the Lebanese American University-School of Medicine. Evidence to support consequential validity was gathered by using criterion-based standard setting methods. Internal structure validity evidence was gathered by examining various psychometric measures both at the station level and across the complete OSCE. Results Compared to our actual method of computing results, the introduction of standard setting resulted in lower students’ average grades and a higher cut score. Across stations, Cronbach’s alpha was moderately low. Conclusion Gathering consequential and internal structure validity evidence by multiple metrics provides support for or against the quality of an OSCE. It is critical that this analysis be performed routinely on local iterations of given tests, and the results used to enhance the quality of assessment.http://link.springer.com/article/10.1186/s12909-018-1421-xObjective structured clinical examinationValidity evidenceQuality assurance |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Vanda Yazbeck Karam Yoon Soo Park Ara Tekian Nazih Youssef |
spellingShingle |
Vanda Yazbeck Karam Yoon Soo Park Ara Tekian Nazih Youssef Evaluating the validity evidence of an OSCE: results from a new medical school BMC Medical Education Objective structured clinical examination Validity evidence Quality assurance |
author_facet |
Vanda Yazbeck Karam Yoon Soo Park Ara Tekian Nazih Youssef |
author_sort |
Vanda Yazbeck Karam |
title |
Evaluating the validity evidence of an OSCE: results from a new medical school |
title_short |
Evaluating the validity evidence of an OSCE: results from a new medical school |
title_full |
Evaluating the validity evidence of an OSCE: results from a new medical school |
title_fullStr |
Evaluating the validity evidence of an OSCE: results from a new medical school |
title_full_unstemmed |
Evaluating the validity evidence of an OSCE: results from a new medical school |
title_sort |
evaluating the validity evidence of an osce: results from a new medical school |
publisher |
BMC |
series |
BMC Medical Education |
issn |
1472-6920 |
publishDate |
2018-12-01 |
description |
Abstract Background To prevent the problems of traditional clinical evaluation, the “Objective Structured Clinical Examination (OSCE)” was presented by Harden as a more valid and reliable assessment instrument. However, an essential condition to guarantee a high-quality and effective OSCE is the assurance of evidence to support the validity of its scores. This study examines the psychometric properties of OSCE scores, with an emphasis on consequential and internal structure validity evidence. Methods Fifty-three first year medical students took part in a summative OSCE at the Lebanese American University-School of Medicine. Evidence to support consequential validity was gathered by using criterion-based standard setting methods. Internal structure validity evidence was gathered by examining various psychometric measures both at the station level and across the complete OSCE. Results Compared to our actual method of computing results, the introduction of standard setting resulted in lower students’ average grades and a higher cut score. Across stations, Cronbach’s alpha was moderately low. Conclusion Gathering consequential and internal structure validity evidence by multiple metrics provides support for or against the quality of an OSCE. It is critical that this analysis be performed routinely on local iterations of given tests, and the results used to enhance the quality of assessment. |
topic |
Objective structured clinical examination Validity evidence Quality assurance |
url |
http://link.springer.com/article/10.1186/s12909-018-1421-x |
work_keys_str_mv |
AT vandayazbeckkaram evaluatingthevalidityevidenceofanosceresultsfromanewmedicalschool AT yoonsoopark evaluatingthevalidityevidenceofanosceresultsfromanewmedicalschool AT aratekian evaluatingthevalidityevidenceofanosceresultsfromanewmedicalschool AT nazihyoussef evaluatingthevalidityevidenceofanosceresultsfromanewmedicalschool |
_version_ |
1724512507423358976 |