Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation
The stabilization and validation process of the measured position of objects is an important step for high-level perception functions and for the correct processing of sensory data. The goal of this process is to detect and handle inconsistencies between different sensor measurements, which result f...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2020-02-01
|
Series: | Sensors |
Subjects: | |
Online Access: | https://www.mdpi.com/1424-8220/20/4/1110 |
id |
doaj-fd222998ec014d23b2f23c6882af37fb |
---|---|
record_format |
Article |
spelling |
doaj-fd222998ec014d23b2f23c6882af37fb2020-11-25T02:03:24ZengMDPI AGSensors1424-82202020-02-01204111010.3390/s20041110s20041110Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic SegmentationMircea Paul Muresan0Ion Giosan1Sergiu Nedevschi2Computer Science Department, Technical University of Cluj-Napoca, 28 Memorandumului Street, 400114 Cluj Napoca, RomaniaComputer Science Department, Technical University of Cluj-Napoca, 28 Memorandumului Street, 400114 Cluj Napoca, RomaniaComputer Science Department, Technical University of Cluj-Napoca, 28 Memorandumului Street, 400114 Cluj Napoca, RomaniaThe stabilization and validation process of the measured position of objects is an important step for high-level perception functions and for the correct processing of sensory data. The goal of this process is to detect and handle inconsistencies between different sensor measurements, which result from the perception system. The aggregation of the detections from different sensors consists in the combination of the sensorial data in one common reference frame for each identified object, leading to the creation of a super-sensor. The result of the data aggregation may end up with errors such as false detections, misplaced object cuboids or an incorrect number of objects in the scene. The stabilization and validation process is focused on mitigating these problems. The current paper proposes four contributions for solving the stabilization and validation task, for autonomous vehicles, using the following sensors: trifocal camera, fisheye camera, long-range RADAR (Radio detection and ranging), and 4-layer and 16-layer LIDARs (Light Detection and Ranging). We propose two original data association methods used in the sensor fusion and tracking processes. The first data association algorithm is created for tracking LIDAR objects and combines multiple appearance and motion features in order to exploit the available information for road objects. The second novel data association algorithm is designed for trifocal camera objects and has the objective of finding measurement correspondences to sensor fused objects such that the super-sensor data are enriched by adding the semantic class information. The implemented trifocal object association solution uses a novel polar association scheme combined with a decision tree to find the best hypothesis−measurement correlations. Another contribution we propose for stabilizing object position and unpredictable behavior of road objects, provided by multiple types of complementary sensors, is the use of a fusion approach based on the Unscented Kalman Filter and a single-layer perceptron. The last novel contribution is related to the validation of the 3D object position, which is solved using a fuzzy logic technique combined with a semantic segmentation image. The proposed algorithms have a real-time performance, achieving a cumulative running time of 90 ms, and have been evaluated using ground truth data extracted from a high-precision GPS (global positioning system) with 2 cm accuracy, obtaining an average error of 0.8 m.https://www.mdpi.com/1424-8220/20/4/1110data associationmulti-object trackingsensor fusionmotion compensationneural networks |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Mircea Paul Muresan Ion Giosan Sergiu Nedevschi |
spellingShingle |
Mircea Paul Muresan Ion Giosan Sergiu Nedevschi Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation Sensors data association multi-object tracking sensor fusion motion compensation neural networks |
author_facet |
Mircea Paul Muresan Ion Giosan Sergiu Nedevschi |
author_sort |
Mircea Paul Muresan |
title |
Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation |
title_short |
Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation |
title_full |
Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation |
title_fullStr |
Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation |
title_full_unstemmed |
Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation |
title_sort |
stabilization and validation of 3d object position using multimodal sensor fusion and semantic segmentation |
publisher |
MDPI AG |
series |
Sensors |
issn |
1424-8220 |
publishDate |
2020-02-01 |
description |
The stabilization and validation process of the measured position of objects is an important step for high-level perception functions and for the correct processing of sensory data. The goal of this process is to detect and handle inconsistencies between different sensor measurements, which result from the perception system. The aggregation of the detections from different sensors consists in the combination of the sensorial data in one common reference frame for each identified object, leading to the creation of a super-sensor. The result of the data aggregation may end up with errors such as false detections, misplaced object cuboids or an incorrect number of objects in the scene. The stabilization and validation process is focused on mitigating these problems. The current paper proposes four contributions for solving the stabilization and validation task, for autonomous vehicles, using the following sensors: trifocal camera, fisheye camera, long-range RADAR (Radio detection and ranging), and 4-layer and 16-layer LIDARs (Light Detection and Ranging). We propose two original data association methods used in the sensor fusion and tracking processes. The first data association algorithm is created for tracking LIDAR objects and combines multiple appearance and motion features in order to exploit the available information for road objects. The second novel data association algorithm is designed for trifocal camera objects and has the objective of finding measurement correspondences to sensor fused objects such that the super-sensor data are enriched by adding the semantic class information. The implemented trifocal object association solution uses a novel polar association scheme combined with a decision tree to find the best hypothesis−measurement correlations. Another contribution we propose for stabilizing object position and unpredictable behavior of road objects, provided by multiple types of complementary sensors, is the use of a fusion approach based on the Unscented Kalman Filter and a single-layer perceptron. The last novel contribution is related to the validation of the 3D object position, which is solved using a fuzzy logic technique combined with a semantic segmentation image. The proposed algorithms have a real-time performance, achieving a cumulative running time of 90 ms, and have been evaluated using ground truth data extracted from a high-precision GPS (global positioning system) with 2 cm accuracy, obtaining an average error of 0.8 m. |
topic |
data association multi-object tracking sensor fusion motion compensation neural networks |
url |
https://www.mdpi.com/1424-8220/20/4/1110 |
work_keys_str_mv |
AT mirceapaulmuresan stabilizationandvalidationof3dobjectpositionusingmultimodalsensorfusionandsemanticsegmentation AT iongiosan stabilizationandvalidationof3dobjectpositionusingmultimodalsensorfusionandsemanticsegmentation AT sergiunedevschi stabilizationandvalidationof3dobjectpositionusingmultimodalsensorfusionandsemanticsegmentation |
_version_ |
1724948555054972928 |