Study on appropriateness of interrater chance-corrected agreement coefficients

• Main Authors: Yan-Ling Kuo, 郭晏伶
• Other Authors: Hung Chen
• Format: Others
• Language: zh-TW
• Published: 2012
• Online Access: http://ndltd.ncl.edu.tw/handle/86180881787759289329

碩士 === 國立臺灣大學 === 數學研究所 === 100 === On behavioural research applications, it often needs to quantify the homogeneity of agreement between responses given by two (or more) raters or between two (or more) measurement devices. For a given object, it can receive different ratings from different raters. The reliability among raters becomes an important issue. In particular, investigators would like to know whether all raters classify objects in a consistent manner. Cohen (1960) proposed kappa coefficient, κ, for correcting the chance agreement among two raters. κ is widely used in literature for quantifying agreement among the raters on a nominal scale. However, Cohen''s kappa coefficient has been criticized for the illness prevalence or base rate in the particular population under study or irrelevant of rater''s rating abilities for latent classes. Gwet (2008) proposed an alternative agreement based on interrater reliability called AC1 statistic, γ1. De Mast (2007) suggested an appropriate chance-corrected interrater agreement coefficient κ* by correcting the agreement due to chance. In this thesis, we use asymptotic analysis to evaluate whether κ or γ1 is a consistent estimate of κ* when both raters adopt random rating model or Gwet''s model (2008) and compare the performances of κ and γ1 with κ*.