The Development and Evaluation of a Gaze-aware Real-time Translation Tool

• Main Authors: Yi-Hsuan Lee, 李宜軒
• Other Authors: Yei-Yu Yeh
• Format: Others
• Language: zh-TW
• Published: 2017
• Online Access: http://ndltd.ncl.edu.tw/handle/y53gt4

碩士 === 國立臺灣大學 === 心理學研究所 === 105 === Non-native speakers often need instant translation for comprehending documents in foreign language. To meet this need, prior research has attempted to map users’ eye movement to cognitive processes for recognizing their intention in real-time and improving the reading experience. However, recent research in gaze-based interaction mostly used fixed dwell-time as the threshold to determine whether or not to provide assistance. This approach ignores individual differences in reading behavior. Moreover, the benefits of gaze-contingent feedback were based on subjective evaluation so that the impact on objective performance is still missing. This study developed a gaze-aware instant translation prototype based on the psychological aspects of eye movements in reading by incorporating a mathematical model of eye-movement control in reading for computing fixation durations as a function of lexical difficulty. The system automatically provided Chinese translations or a meaningless X mask based on dwell-time or regressive saccades to investigate how the annotations triggered by deviation in reading pattern affect a user’s eye movements, cognitive load and comprehension accuracy. The results showed that proactive translations could improve reading comprehension and reduce cognitive load. Instant annotations also influence a user’s reading strategy, including prolonging fixation durations, decreasing skipping rate and increasing regression rate. Furthermore, the frequency of the target words was related to their fixation patterns. According to global and local eye-movement measures, the annotations provided based on personalized thresholds of dwell time were significantly helpful for word processing. Considering the methodological issues and observed results, this thesis suggests design guidelines and recommendations for developing gaze-aware reading applications. Future research could investigate whether a personalized gaze-aware annotation could be applied to the real-world settings or adapted to different devices.