Deep Q‐network implementation for simulated autonomous vehicle control

• Main Authors: Yang Thee Quek, Li Ling Koh, Ngiap Tiam Koh, Wai Ann Tso, Wai Lok Woo
• Format: Article
• Language: English
• Published: Wiley 2021-07-01
• Series: IET Intelligent Transport Systems
• Online Access: https://doi.org/10.1049/itr2.12067

Abstract Deep reinforcement learning is poised to be a revolutionised step towards newer possibilities in solving navigation and autonomous vehicle control tasks. Deep Q‐network (DQN) is one of the more popular methods of deep reinforcement learning that allows the agent that controls the vehicle to learn through its mistakes based on its actions and interactions with the environment. This paper presents the implementation of DQN to an autonomous self‐driving vehicle control in two different simulated environments; first environment is in Python which is a simple 2D environment and then advanced to Unity software separately which is a 3D environment. Based on the scores and pixel inputs, the agent in the vehicle learns and adapts to its surrounding. It develops the best solution strategy to direct itself in the environment where its task is to manoeuvre the vehicle from point to point on a simulated highway scenario. The implemented DQN technique approximates the action value function with convolutional neural network. This evaluates the Q‐function for the Q‐learning architecture and updates the action value function. This paper shows that DQN is an effective learning method for the agent of an autonomous vehicle. In both simulated environments, the autonomous vehicle gradually learnt the manoeuvre operations and progressively gained the ability to successfully navigate itself and avoid obstacles without prior information of the surrounding.