A Distributed Representation Based Semantic Measure Approach for Automatic After-Reading Comprehension Assessments

• Main Authors: Fu-Yuan Hsu, 許福元
• Other Authors: HAHN-MING LEE
• Format: Others
• Language: en_US
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/vcnww7

博士 === 國立臺灣科技大學 === 資訊工程系 === 106 === Reading ability has become an indispensable ability for modern citizens, and the past few years have seen many countries striving to improve their reading education by actively introducing adaptive reading platforms to improve students' reading comprehension. Adaptive reading platforms must accurately measure students' reading ability, objectively assess the difficulty of books, and quickly assess after-reading comprehension. Only then could they accurately pair students with suitably challenging texts and grasp the dynamic reading situation of students. However, existing adaptive reading platforms have not been able to provide an automatic after-reading assessment utility to reduce the burden on reading teachers. Traditionally, teachers perform after-reading assessments by administering multiple-choice tests or by requesting their students write summaries. However, this means that teachers must design test items or grade summaries for each book that students read. It is a never-ending, arduous task for teachers. To fulfill students' needs for extensive reading, teachers may design test items through collective wisdom or automatic item generation technologies. Regardless, to ensure that the assessment objectives are achieved, it is necessary to control the difficulty of mass-generated items effectively. Currently, this is accomplished mainly by using pretests to get accurate item difficulty, but doing so is laborious, time-consuming, and leaves doubts about the safety of the items. Current summary scoring systems are mainly designed for single topics and short articles, and may not apply well to long texts. To solve these difficulties, this study proposes a semantic similarity measurement based on distributed representations and utilizes the semantic similarity between a stem, an answer, and distractors as the semantic features of item difficulty. Then, through machine learning, an estimation model can be trained that evaluates the difficulty of automatically generated test items. The same measurement method is used in automatic summary scoring: First, the semantic vector of each sentence in the text is calculated, then clustering technology is used to obtain book summaries, which are then used to establish a scoring model for after-reading summaries. Experimental results show that the method proposed in this study has reasonably good performance.