Deep Multimodal Detection in Reduced Visibility Using Thermal Depth Estimation for Autonomous Driving

Recently, the rapid development of convolutional neural networks (CNN) has consistently improved object detection performance using CNN and has naturally been implemented in autonomous driving due to its operational potential in real-time. Detecting moving targets to realize autonomous driving is an...

Full description

Bibliographic Details
Main Authors: Cho, J. (Author), Yoon, S. (Author)
Format: Article
Language:English
Published: MDPI 2022
Subjects:
Online Access:View Fulltext in Publisher
LEADER 03034nam a2200433Ia 4500
001 10.3390-s22145084
008 220718s2022 CNT 000 0 und d
020 |a 14248220 (ISSN) 
245 1 0 |a Deep Multimodal Detection in Reduced Visibility Using Thermal Depth Estimation for Autonomous Driving 
260 0 |b MDPI  |c 2022 
856 |z View Fulltext in Publisher  |u https://doi.org/10.3390/s22145084 
520 3 |a Recently, the rapid development of convolutional neural networks (CNN) has consistently improved object detection performance using CNN and has naturally been implemented in autonomous driving due to its operational potential in real-time. Detecting moving targets to realize autonomous driving is an essential task for the safety of drivers and pedestrians, and CNN-based moving target detectors have shown stable performance in fair weather. However, there is a consider-able drop in detection performance during poor weather conditions like hazy or foggy situations due to particles in the atmosphere. To ensure stable moving object detection, an image restoration process with haze removal must be accompanied. Therefore, this paper proposes an image dehazing network that estimates the current weather conditions and removes haze using the haze level to improve the detection performance under poor weather conditions due to haze and low visibility. Combined with the thermal image, the restored image is assigned to the two You Only Look Once (YOLO) object detectors, respectively, which detect moving targets independently and improve object detection performance using late fusion. The proposed model showed improved dehazing performance compared with the existing image dehazing models and has proved that images taken under foggy conditions, the poorest weather for autonomous driving, can be restored to normal images. Through the fusion of the RGB image restored by the proposed image dehazing network with thermal images, the proposed model improved the detection accuracy by up to 22% or above in a dense haze environment like fog compared with models using existing image dehazing techniques. © 2022 by the authors. Licensee MDPI, Basel, Switzerland. 
650 0 4 |a autonomous driving 
650 0 4 |a Autonomous driving 
650 0 4 |a Autonomous vehicles 
650 0 4 |a Condition 
650 0 4 |a Convolutional neural network 
650 0 4 |a Convolutional neural networks 
650 0 4 |a Dehazing 
650 0 4 |a Demulsification 
650 0 4 |a depth estimation 
650 0 4 |a Depth Estimation 
650 0 4 |a Detection performance 
650 0 4 |a image dehazing 
650 0 4 |a Image dehazing 
650 0 4 |a Image enhancement 
650 0 4 |a Image fusion 
650 0 4 |a Image reconstruction 
650 0 4 |a Moving targets 
650 0 4 |a object detection 
650 0 4 |a Object detection 
650 0 4 |a Object recognition 
650 0 4 |a Objects detection 
650 0 4 |a Restoration 
650 0 4 |a Thermal images 
650 0 4 |a Visibility 
700 1 |a Cho, J.  |e author 
700 1 |a Yoon, S.  |e author 
773 |t Sensors