A Computerized Adaptive Test System for Formative Assessment with Self-Regulation and Signal Theory

• Main Authors: HSU, YU-WEI, 許浴煒
• Other Authors: YING, MING-HSIUNG
• Format: Others
• Language: zh-TW
• Published: 2017
• Online Access: http://ndltd.ncl.edu.tw/handle/jpxp4q

碩士 === 中華大學 === 資訊管理學系 === 105 === In recent years, computer-aided instruction has started to get popular because of the progress of information technology and internet. The computerization of testing process gradually derives computer-aided testing. Studied in the past confirmed that the integration of information technology into instruction could enhance teaching effectiveness as well as contribute to students’ learning interest and learning ability. In view of the many advantages brought by computer technology, teaching platforms based on internet spring up rapidly like mushrooms. Testing is an important part of teaching process and its result can reflect the degree of students’ learning; adaptive testing could achieve the purpose of “testing students in accordance with their aptitude” because it provides questions suitable to students’ ability according to learners’ capacity and responding speed. Current studies for adaptive testing mainly focus on the environment of applying item response theory to summative assessment while this research emphasizes teaching activity application of the course as well as quality management of exam questions. Therefore, the limitation of the traditional adaptive testing that is only applicable to summative assessment is improved and further urges the spirit of CAT to be used in the environment of campus formative evaluation. Summative assessment is a test that provides grades or ranking after the teaching curriculum is completed in order to judge teaching effectiveness. Currently, formative evaluation has become the main testing evaluation model of teaching activity and relevant studies also highlight that it can continue and collect evidences of students’ learning systematically. As a result, this research integrates adaptive testing system with formative evaluation as the important core element of the research. Many scholars in the past applied the concept of self-regulation to the studies for education and firmly believed self-regulation learning is the key to learning success. The element of self-regulation has been put to the research to raise the methods of self-observation and self-judgement by learners so that to stimulate learners’ learning adjustment. Thus, the concept of signal theory from the economics has been added to the system; using different signals for different information in order to stimulate learner’s adjustment on their learning strategies. The research classifies adaptive testing into five levels in order to encourage student’s learning, help students set up their learning target, and keep up with the teaching concept of ability rating commonly used for English in universities. During the test, students with the same level will receive exam papers with similar difficulty of questions but the contents of each paper are different. In order to fulfill the system functions mentioned above, the research proposed a prototype for system structure and its important functions include college subsystem, curriculum subsystem, exam question bank management subsystem, online test subsystem, testing event and question selection subsystem as well as paper-marking and test feedback system. Other than offering the function of online test, the prototype developed by the research also provides hard-copy exam paper printing and 2B answer sheet marking. After adaptive testing, the grades generated by computerized marking will be converted into fair standard scores based on the fairness consideration through internal adaptive score converter mechanism for teachers’ reference and for students’ learning suggestion and weakness analysis as well as for stimulation on learners by testing feedback information so that to achieve the purpose of self-regulation. For further understanding the effectiveness of the adaptive testing system proposed by the research, a survey of learning interest was conducted before and after the test. Through the verification of Paired-Samples T Test, it approved the system has outstanding improvement on learning interest. In addition, in terms of pass rate of ERP certificate, the course used the same teacher and the same teaching materials in academic year 2014 and 2015; the students’ level was considerably close. The traditional classical test of the same exam paper for everybody was used for academic year 2014 while the adaptive pencil-and-paper test based on testing students in accordance with their aptitude was used in 2015. A comparison of ERP on-campus certification test by the fair third party revealed the pass rate in academic year 2015 has increased 16.04%. An inference of the system of the research can improve the pass rate of ERP certificate has been made. As for the satisfaction of system, both male and female students are with very high satisfaction towards the system of the research.