Video-Based Human Motion Capture Data Retrieval via MotionSet Network

• Main Authors: Tingxin Ren, Wei Li, Zifei Jiang, Xueqing Li, Yan Huang, Jingliang Peng
• Format: Article
• Language: English
• Published: IEEE 2020-01-01
• Series: IEEE Access
• Subjects: MotionSet; motion capture data retrieval; convolutional neural network; deep learning
• Online Access: https://ieeexplore.ieee.org/document/9220910/

Content-based human motion capture (MoCap) data retrieval facilitates reusing motion data that have already been captured and stored in a database. For a MoCap data retrieval system to get practically deployed, both high precision and natural interface are demanded. Targeting both, we propose a video-based human MoCap data retrieval solution in this work. It lets users to specify a query via a video clip, addresses the representational gap between video and MoCap clips and extracts discriminative motion features for precise retrieval. Specifically, the proposed scheme firstly converts each video clip or MoCap clip at a certain viewpoint to a binary silhouette sequence. Regarding a video or MoCap clip as a set of silhouette images, the proposed scheme uses a convolutional neural network, named MotionSet, to extract the discriminative motion feature of the clip. The extracted motion features are used to match a query to repository MoCap clips for the retrieval. Besides the algorithmic solution, we also contribute a human MoCap dataset and a human motion video dataset in couple that contain various action classes. Experiments show that our proposed scheme achieves an increase of around 0.25 in average MAP and costs about 1/26 time for online retrieval, when compared with the benchmark algorithm.