Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education Students
This study aimed to examine the effect of rater training on the differential rater function (rater error) in the process of assessing the academic writing skills of higher education students. The study was conducted with a pre-test and post-test control group quasi-experimental design. The study g...
Main Authors: | , |
---|---|
Format: | Article |
Language: | English |
Published: |
EPODDER
2021-06-01
|
Series: | Journal of Measurement and Evaluation in Education and Psychology |
Subjects: | |
Online Access: | https://dergipark.org.tr/en/pub/epod/issue/63232/842094 |
id |
doaj-cdea79694266471ebc327b8f5552832f |
---|---|
record_format |
Article |
spelling |
doaj-cdea79694266471ebc327b8f5552832f2021-08-09T07:29:15ZengEPODDERJournal of Measurement and Evaluation in Education and Psychology1309-65751309-65752021-06-0112216318110.21031/epod.842094Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education StudentsMehmet ŞATA0https://orcid.org/0000-0003-2683-4997İsmail KARAKAYA1https://orcid.org/0000-0003-4308-6919Agri Ibrahim Cecen University Gazi UniversityThis study aimed to examine the effect of rater training on the differential rater function (rater error) in the process of assessing the academic writing skills of higher education students. The study was conducted with a pre-test and post-test control group quasi-experimental design. The study group of the research consisted of 45 raters, of whom 22 came from experimental, and 23 came from control groups. The raters were pre-service teachers who did not participate in any rater training before, and it was investigated that they had similar experiences in assessment. The data were collected using an analytical rubric developed by the researchers and an opinion-based writing task prepared by the International English Language Testing System (IELTS). Within the scope of the research, the compositions of 39 students that were written in a foreign language (English) were assessed. Many Facet Rasch Model was used for the analysis of the data, and this analysis was conducted under the Fully Crossed Design. The findings of the study revealed that the given rater training was effective on differential rater function, and suggestions based on these results were presented.https://dergipark.org.tr/en/pub/epod/issue/63232/842094academic writingmany facet rasch modelrater trainingdifferential rater function |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Mehmet ŞATA İsmail KARAKAYA |
spellingShingle |
Mehmet ŞATA İsmail KARAKAYA Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education Students Journal of Measurement and Evaluation in Education and Psychology academic writing many facet rasch model rater training differential rater function |
author_facet |
Mehmet ŞATA İsmail KARAKAYA |
author_sort |
Mehmet ŞATA |
title |
Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education Students |
title_short |
Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education Students |
title_full |
Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education Students |
title_fullStr |
Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education Students |
title_full_unstemmed |
Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education Students |
title_sort |
investigating the effect of rater training on differential rater function in assessing academic writing skills of higher education students |
publisher |
EPODDER |
series |
Journal of Measurement and Evaluation in Education and Psychology |
issn |
1309-6575 1309-6575 |
publishDate |
2021-06-01 |
description |
This study aimed to examine the effect of rater training on the differential rater function (rater error) in the
process of assessing the academic writing skills of higher education students. The study was conducted with a
pre-test and post-test control group quasi-experimental design. The study group of the research consisted of 45
raters, of whom 22 came from experimental, and 23 came from control groups. The raters were pre-service
teachers who did not participate in any rater training before, and it was investigated that they had similar
experiences in assessment. The data were collected using an analytical rubric developed by the researchers and
an opinion-based writing task prepared by the International English Language Testing System (IELTS). Within
the scope of the research, the compositions of 39 students that were written in a foreign language (English) were
assessed. Many Facet Rasch Model was used for the analysis of the data, and this analysis was conducted under
the Fully Crossed Design. The findings of the study revealed that the given rater training was effective on
differential rater function, and suggestions based on these results were presented. |
topic |
academic writing many facet rasch model rater training differential rater function |
url |
https://dergipark.org.tr/en/pub/epod/issue/63232/842094 |
work_keys_str_mv |
AT mehmetsata investigatingtheeffectofratertrainingondifferentialraterfunctioninassessingacademicwritingskillsofhighereducationstudents AT ismailkarakaya investigatingtheeffectofratertrainingondifferentialraterfunctioninassessingacademicwritingskillsofhighereducationstudents |
_version_ |
1721215137751236608 |