| Summary: | Classification testing has been widely used to make categorical decisions by determining whether an examinee has a certain degree of ability required by established standards. As computer technologies have developed, classification testing has become more computerized. Several approaches have been proposed and investigated in the context of computer-based classification testing, including: 1) Computerized adaptive test (CAT); 2) Multistage test (MST); 3) Sequential probability ratio test (SPRT), among others.
The purpose of this study was to systematically compare the differences in classification decision precision among several testing approaches (i.e., CAT, MST, and SPRT) given three test lengths and three cutoff scores using mixed-format tests based on the generalized partial credit model. The progressive-restricted exposure control procedure and constrained CAT content balancing procedure with test unit types were also incorporated as part of this study. All conditions were evaluated in terms of the classification decision precision and the exposure control property.
Overall, this study’s results indicated that all three approaches performed well in terms of classifying people into two categories. The CAT and SPRT approaches produced, on average, comparable results with both performing relatively better than the MST approach in the precision of their classification decision. As the test length increased, the classification decision accuracy generally increased for all approaches; however, the CAT and SPRT approaches yielded more accuracy with the shorter test length. In terms of cutoff scores, predicting classification decision differed according to the location of cutoff scores based on the normal distribution of examinees.
In terms of exposure control properties, the progressive-restricted exposure control procedure with the pre-set maximum test unit exposure rate was implemented effectively into the CAT and SPRT approaches. The CAT approach had, on average, a higher proportion of test units with low test unit exposure rates and produced better results in pool utilization rates than the SPRT approach. Finally, the MST approach administered all test units constructed for the panels for each condition. It had, on average, however, a higher proportion of test units with high test unit exposure rates because computations were based only on the proportion of whole test unit pool used for constructing the MST panels. === text
|
|---|
