A Semantic-Based Gas Source Localization with a Mobile Robot Combining Vision and Chemical Sensing

• Main Authors: Javier Monroy, Jose-Raul Ruiz-Sarmiento, Francisco-Angel Moreno, Francisco Melendez-Fernandez, Cipriano Galindo, Javier Gonzalez-Jimenez
• Format: Article
• Language: English
• Published: MDPI AG 2018-11-01
• Series: Sensors
• Subjects: e-nose; electronic nose; gas sensor; mobile robot; gas source localization; semantics; uncertainty; sensor fusion; object recognition; MDP; Markov decision process; gas classification; Bayes; uncertainty propagation; Mobile Robot Olfaction; chemical sensors; Artificial Intelligence; Machine Learning
• Online Access: https://www.mdpi.com/1424-8220/18/12/4174

This paper addresses the localization of a gas emission source within a real-world human environment with a mobile robot. Our approach is based on an efficient and coherent system that fuses different sensor modalities (i.e., vision and chemical sensing) to exploit, for the first time, the semantic relationships among the detected gases and the objects visually recognized in the environment. This novel approach allows the robot to focus the search on a finite set of potential gas source candidates (dynamically updated as the robot operates), while accounting for the non-negligible uncertainties in the object recognition and gas classification tasks involved in the process. This approach is particularly interesting for structured indoor environments containing multiple obstacles and objects, enabling the inference of the relations between objects and between objects and gases. A probabilistic Bayesian framework is proposed to handle all these uncertainties and semantic relations, providing an ordered list of candidates to be the source. This candidate list is updated dynamically upon new sensor measurements to account for objects not previously considered in the search process. The exploitation of such probabilities together with information such as the locations of the objects, or the time needed to validate whether a given candidate is truly releasing gases, is delegated to a path planning algorithm based on Markov decision processes to minimize the search time. The system was tested in an office-like scenario, both with simulated and real experiments, to enable the comparison of different path planning strategies and to validate its efficiency under real-world conditions.