Application of OpenPose and deep learning for intelligent surveillance reconnaissance system

• Main Authors: Kyujung Choi, Suyeong Oh, Chaebong Sohn
• Format: Article
• Language: English
• Published: Institute of Defense Acquisition Program 2020-12-01
• Series: 선진국방연구
• Subjects: OpenPose; keypoints; deep neural networks; convolutional neural networks; long short-term memory
• Online Access: https://journal.idap.re.kr/index.php/JAMS/article/view/80
• ISSN: 2635-5531; 2636-1329

In this study, defense surveillance reconnaissance systems were implemented through deep learning networks such as OpenPose and deep neural networks (DNN), convolutional neural networks (CNN), and long short-term memory (LSTM). This study proposes a target recognition method which differs from the existing surveillance reconnaissance systems. This method consists in distinguishing between ordinary people and targets by classifying motions in the images being filmed. Thus, the skeleton data of the target in the image are extracted using OpenPose. Then, keypoints included in the extracted skeleton data are entered into DNN, CNN, and LSTM to classify the motion. The classified motions are selected as motions learned in the military, such as overall security. When the system classifies motions and recognizes targets, it identifies them on the map and tracks them. The tracking algorithm calculates the movement direction of the target by calculating the change in the values of keypoints extracted through OpenPose by frames. Finally, it uses the depth information obtained from the camera to display targets on the map based on the camera location. All these computations are based on the use of the skeleton data rather than the entire image, thus reducing the overall computation.