Reasoning for Local Graph Over Knowledge Graph With a Multi-Policy Agent

• Main Authors: Jie Zhang, Zhongmin Pei, Zhangkai Luo
• Format: Article
• Language: English
• Published: IEEE 2021-01-01
• Series: IEEE Access
• Subjects: Knowledge graph completion; local graph reasoning; Markov decision process; reinforcement learning; searching window
• Online Access: https://ieeexplore.ieee.org/document/9440923/

The reinforcement learning framework for multi-hop relational paths is one of the effective methods for solving knowledge graph incompletion. However, these models are associated with limited performances attributed to delayed rewards and far-fetched search trajectories. To overcome these challenges, we propose the searching window and multi-policy agent. The searching window provides a large action space, so that the agent can backtrack based on the newly obtained information and establish a local graph instead of a path chain. Based on the searching window, a double long short-term memory (DBL-LSTM) policy network is introduced to encode the local graph and relation sequence, after which the encoding information is used by the agent to select a correct entity to grow the local graph. Furthermore, multi-policy agent separately infers the local graph through three different policy networks, then, all local graphs are integrated into an information-rich local graph. Experiments using the WN18RR dataset revealed that local graph reasoning with searching window had greater rewards than path reasoning, the proposed DBL-LSTM policy network improved all HITS@N(N = 1,3,5,10) compared to prior works, and that the multi-policy agent achieved higher hit rates than single-policy agent.