Individual ability on high-stakes test: Choosing cumulative score or rasch for scoring model

• Published in: Jurnal Penelitian dan Evaluasi Pendidikan
• Main Authors: Muhammad Dhiyaul Khair, Sukaesi Marianti
• Format: Article
• Language: English
• Published: Universitas Negeri Yogyakarta 2024-06-01
• Subjects: ability; cumulative score; rasch; dichotomous responses
• Online Access: https://journal.uny.ac.id/index.php/jpep/article/view/71661

In a test, a method is required to estimate an individual's ability based on their responses. Typically, this is done by summing the correct responses or calculating a cumulative score. An alternative method is the Rasch model. This study aims to determine whether an individual's position, based on cumulative score estimates, remains unchanged or changes when compared with ability estimates using Rasch on dichotomous responses. The study uses open-source data from the 2018 Program for International Student Assessment (PISA) by the Organization for Economic Co-operation and Development (OECD) and involves 317 Indonesian students.  Ability analysis will be conducted on Math and Reading aspects using cumulative scores and Rasch with dichotomous responses. The study will employ data analysis techniques such as Rasch, paired samples t-test, and descriptive statistical analysis. The cumulative score and Rasch results will be tested using a paired samples t-test, and a comparison of the cumulative score and Rasch estimation results will be carried out using descriptive statistical analysis. The study results indicate that there are differences in individual positions based on ability estimates using cumulative score and Rasch. These differences are caused by variations in scores. Therefore, even if two individuals have the same cumulative score, they may have different Rasch estimates.