Exploring Deep Recurrent Q-Learning for Navigation in a 3D Environment

Learning to navigate in 3D environments from raw sensory input is an important step towards bridging the gap between human players and artificial intelligence in digital games. Recent advances in deep reinforcement learning have seen success in teaching agents to play Atari 2600 games from raw pixel...

Full description

Bibliographic Details
Main Authors: Rasmus Kongsmar Brejl, Henrik Purwins, Henrik Schoenau-Fog
Format: Article
Language:English
Published: European Alliance for Innovation (EAI) 2018-01-01
Series:EAI Endorsed Transactions on Creative Technologies
Subjects:
Online Access:http://eudl.eu/doi/10.4108/eai.16-1-2018.153641
id doaj-bb7587eb43cf428993e3a7e21a149086
record_format Article
spelling doaj-bb7587eb43cf428993e3a7e21a1490862020-11-25T02:02:37ZengEuropean Alliance for Innovation (EAI)EAI Endorsed Transactions on Creative Technologies2409-97082018-01-015141510.4108/eai.16-1-2018.153641Exploring Deep Recurrent Q-Learning for Navigation in a 3D EnvironmentRasmus Kongsmar Brejl0Henrik Purwins1Henrik Schoenau-Fog2The Center for Applied Game Research, Department of Architecture, Design, and Media Technology, Technical Faculty of IT and Design, Aalborg University Copenhagen, Denmark; Audio Analysis Lab, Department of Architecture, Design, and Media Technology, Technical Faculty of IT and Design, Aalborg University Copenhagen, Denmark; rasmuskbrejl@gmail.com The Center for Applied Game Research, Department of Architecture, Design, and Media Technology, Technical Faculty of IT and Design, Aalborg University Copenhagen, Denmark; Audio Analysis Lab, Department of Architecture, Design, and Media Technology, Technical Faculty of IT and Design, Aalborg University Copenhagen, DenmarkThe Center for Applied Game Research, Department of Architecture, Design, and Media Technology, Technical Faculty of IT and Design, Aalborg University Copenhagen, DenmarkLearning to navigate in 3D environments from raw sensory input is an important step towards bridging the gap between human players and artificial intelligence in digital games. Recent advances in deep reinforcement learning have seen success in teaching agents to play Atari 2600 games from raw pixel information where the environment is always fully observable by the agent. This is not true for first-person 3D navigation tasks. Instead, the agent is limited by its field of view which limits its ability to make optimal decisions in the environment. This paper explores using a Deep Recurrent Q-Network implementation with a long short-term memory layer for dealing with such tasks by allowing an agent to process recent frames and gain a memory of the environment. An agent was trained in a 3D first-person labyrinth-like environment for 2 million frames. Informal observations indicate that the trained agent navigated in the right direction but was unable to find the target of the environment.http://eudl.eu/doi/10.4108/eai.16-1-2018.153641Reinforcement LearningDeep LearningQ-LearningDeep Recurrent Q-LearningArtificial IntelligenceNavigationGame Intelligence
collection DOAJ
language English
format Article
sources DOAJ
author Rasmus Kongsmar Brejl
Henrik Purwins
Henrik Schoenau-Fog
spellingShingle Rasmus Kongsmar Brejl
Henrik Purwins
Henrik Schoenau-Fog
Exploring Deep Recurrent Q-Learning for Navigation in a 3D Environment
EAI Endorsed Transactions on Creative Technologies
Reinforcement Learning
Deep Learning
Q-Learning
Deep Recurrent Q-Learning
Artificial Intelligence
Navigation
Game Intelligence
author_facet Rasmus Kongsmar Brejl
Henrik Purwins
Henrik Schoenau-Fog
author_sort Rasmus Kongsmar Brejl
title Exploring Deep Recurrent Q-Learning for Navigation in a 3D Environment
title_short Exploring Deep Recurrent Q-Learning for Navigation in a 3D Environment
title_full Exploring Deep Recurrent Q-Learning for Navigation in a 3D Environment
title_fullStr Exploring Deep Recurrent Q-Learning for Navigation in a 3D Environment
title_full_unstemmed Exploring Deep Recurrent Q-Learning for Navigation in a 3D Environment
title_sort exploring deep recurrent q-learning for navigation in a 3d environment
publisher European Alliance for Innovation (EAI)
series EAI Endorsed Transactions on Creative Technologies
issn 2409-9708
publishDate 2018-01-01
description Learning to navigate in 3D environments from raw sensory input is an important step towards bridging the gap between human players and artificial intelligence in digital games. Recent advances in deep reinforcement learning have seen success in teaching agents to play Atari 2600 games from raw pixel information where the environment is always fully observable by the agent. This is not true for first-person 3D navigation tasks. Instead, the agent is limited by its field of view which limits its ability to make optimal decisions in the environment. This paper explores using a Deep Recurrent Q-Network implementation with a long short-term memory layer for dealing with such tasks by allowing an agent to process recent frames and gain a memory of the environment. An agent was trained in a 3D first-person labyrinth-like environment for 2 million frames. Informal observations indicate that the trained agent navigated in the right direction but was unable to find the target of the environment.
topic Reinforcement Learning
Deep Learning
Q-Learning
Deep Recurrent Q-Learning
Artificial Intelligence
Navigation
Game Intelligence
url http://eudl.eu/doi/10.4108/eai.16-1-2018.153641
work_keys_str_mv AT rasmuskongsmarbrejl exploringdeeprecurrentqlearningfornavigationina3denvironment
AT henrikpurwins exploringdeeprecurrentqlearningfornavigationina3denvironment
AT henrikschoenaufog exploringdeeprecurrentqlearningfornavigationina3denvironment
_version_ 1724951837258285056