A Study of Bi-directional Recurrent Neural Network on Word Prediction Capability

• Main Authors: Chun-Ta Lin, 林俊達
• Other Authors: Bor-Shen Lin
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/2u7k98

碩士 === 國立臺灣科技大學 === 資訊管理系 === 106 === The aim of this research is to predict the missing word in a sentence, using the sentence prediction model based on recurrent neural network. A common approach to this problem is N-gram language model, but the weakness of this model is that it can’t use the long-term word relationship in a sentence. In the self prediction experiment of our forward model, we find that except in the front of the sentence, other position in sentence can be predicted to itself very well. And we do the same experiment on backward model, it has the similar attribute with the forward model, Hence, we propose to combine the forward and backward model’s result, to balance the strengths and weakness between two models. We also observe the ability of this model in application of missing word prediction. We can use this method in the article writing word prompt or refine a non-smoothing sentence. During the testing, we find that this model can predict some related and rationality words which can be put appropriately in the sentence. Finally, we compare to the bigram and trigram language model in the missing word prediction experiment, the result shows that our word prediction model is better than the bigram and trigram language model.