An RNN Model for Generating Sentences with a Desired Word at a Desired Position

• Main Authors: Tianbao Song, Jingbo Sun, Yinbing Zhang, Weiming Peng*, Jihua Song
• Format: Article
• Language: English
• Published: Faculty of Mechanical Engineering in Slavonski Brod, Faculty of Electrical Engineering in Osijek, Faculty of Civil Engineering in Osijek 2020-01-01
• Series: Tehnički Vjesnik
• Subjects: desired word; lexically constrained; RNN; sentence generation
• Online Access: https://hrcak.srce.hr/file/340463

Generating sentences with a desired word is useful in many natural language processing tasks. State-of-the-art recurrent neural network (RNN)-based models mainly generate sentences in a left-to-right manner, which does not allow explicit and direct constraints on the words at arbitrary positions in a sentence. To address this issue, we propose a generative model of sentences named Coupled-RNN. We employ two RNN's to generate sentences backwards and forwards respectively starting from a desired word, and inject position embeddings into the model to solve the problem of position information loss. We explore two coupling mechanisms to optimize the reconstruction loss globally. Experimental results demonstrate that Coupled-RNN can generate high quality sentences that contain a desired word at a desired position.