Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences
The early recognition and understanding of the actions performed by pedestrians in traffic scenes leads to an anticipation of pedestrian intentions in advance and helps in the process of collision warning and avoidance in the context of autonomous vehicles. An environment with low visibility conditi...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2021-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9432859/ |
id |
doaj-c4993ac3403a45d79a2b55eca2678a4b |
---|---|
record_format |
Article |
spelling |
doaj-c4993ac3403a45d79a2b55eca2678a4b2021-06-02T23:17:59ZengIEEEIEEE Access2169-35362021-01-019743027432410.1109/ACCESS.2021.30808229432859Pedestrian Street-Cross Action Recognition in Monocular Far Infrared SequencesRaluca Didona Brehar0https://orcid.org/0000-0003-0978-7826Mircea Paul Muresan1https://orcid.org/0000-0003-0315-3507Tiberiu Marita2https://orcid.org/0000-0002-5987-9174Cristian-Cosmin Vancea3https://orcid.org/0000-0003-0270-1962Mihai Negru4https://orcid.org/0000-0001-8039-8415Sergiu Nedevschi5https://orcid.org/0000-0003-2018-4647Department of Computer Science, Technical University of Cluj-Napoca, Cluj-Napoca, RomaniaDepartment of Computer Science, Technical University of Cluj-Napoca, Cluj-Napoca, RomaniaDepartment of Computer Science, Technical University of Cluj-Napoca, Cluj-Napoca, RomaniaDepartment of Computer Science, Technical University of Cluj-Napoca, Cluj-Napoca, RomaniaDepartment of Computer Science, Technical University of Cluj-Napoca, Cluj-Napoca, RomaniaDepartment of Computer Science, Technical University of Cluj-Napoca, Cluj-Napoca, RomaniaThe early recognition and understanding of the actions performed by pedestrians in traffic scenes leads to an anticipation of pedestrian intentions in advance and helps in the process of collision warning and avoidance in the context of autonomous vehicles. An environment with low visibility conditions such as night-time, fog, heavy rain or smoke increases the number of difficult situations in traffic. A complete and original model for assessing if a pedestrian is engaged in a street cross action using only infrared monocular scene perception is proposed in this paper. The assessment of a street cross action is done by the time series analysis of features like: pedestrian motion, position of pedestrians with respect to the drivable area and their distance with respect to the ego-vehicle. The extraction of these features emerges from the combination of a deep learning based pedestrian detector with an original tracking algorithm, a semantic segmentation of the road surface and a time series long-short term memory network based action recognition. In order to validate the proposed method we introduce a new dataset named CROSSIR. It is formed of pedestrian annotations, action annotations and semantic labels for the road. The CROSSIR dataset is suitable for several common computer vision algorithms: (1) pedestrian detection and tracking algorithms because each pedestrian has a unique identifier over the frames in which it appears; (2) pedestrian action recognition; (3) semantic segmentation of the road pixels in the infrared image.https://ieeexplore.ieee.org/document/9432859/Image processingneural networkpattern recognitionnight vision applicationsFLIR camerapedestrian detection |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Raluca Didona Brehar Mircea Paul Muresan Tiberiu Marita Cristian-Cosmin Vancea Mihai Negru Sergiu Nedevschi |
spellingShingle |
Raluca Didona Brehar Mircea Paul Muresan Tiberiu Marita Cristian-Cosmin Vancea Mihai Negru Sergiu Nedevschi Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences IEEE Access Image processing neural network pattern recognition night vision applications FLIR camera pedestrian detection |
author_facet |
Raluca Didona Brehar Mircea Paul Muresan Tiberiu Marita Cristian-Cosmin Vancea Mihai Negru Sergiu Nedevschi |
author_sort |
Raluca Didona Brehar |
title |
Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences |
title_short |
Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences |
title_full |
Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences |
title_fullStr |
Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences |
title_full_unstemmed |
Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences |
title_sort |
pedestrian street-cross action recognition in monocular far infrared sequences |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2021-01-01 |
description |
The early recognition and understanding of the actions performed by pedestrians in traffic scenes leads to an anticipation of pedestrian intentions in advance and helps in the process of collision warning and avoidance in the context of autonomous vehicles. An environment with low visibility conditions such as night-time, fog, heavy rain or smoke increases the number of difficult situations in traffic. A complete and original model for assessing if a pedestrian is engaged in a street cross action using only infrared monocular scene perception is proposed in this paper. The assessment of a street cross action is done by the time series analysis of features like: pedestrian motion, position of pedestrians with respect to the drivable area and their distance with respect to the ego-vehicle. The extraction of these features emerges from the combination of a deep learning based pedestrian detector with an original tracking algorithm, a semantic segmentation of the road surface and a time series long-short term memory network based action recognition. In order to validate the proposed method we introduce a new dataset named CROSSIR. It is formed of pedestrian annotations, action annotations and semantic labels for the road. The CROSSIR dataset is suitable for several common computer vision algorithms: (1) pedestrian detection and tracking algorithms because each pedestrian has a unique identifier over the frames in which it appears; (2) pedestrian action recognition; (3) semantic segmentation of the road pixels in the infrared image. |
topic |
Image processing neural network pattern recognition night vision applications FLIR camera pedestrian detection |
url |
https://ieeexplore.ieee.org/document/9432859/ |
work_keys_str_mv |
AT ralucadidonabrehar pedestrianstreetcrossactionrecognitioninmonocularfarinfraredsequences AT mirceapaulmuresan pedestrianstreetcrossactionrecognitioninmonocularfarinfraredsequences AT tiberiumarita pedestrianstreetcrossactionrecognitioninmonocularfarinfraredsequences AT cristiancosminvancea pedestrianstreetcrossactionrecognitioninmonocularfarinfraredsequences AT mihainegru pedestrianstreetcrossactionrecognitioninmonocularfarinfraredsequences AT sergiunedevschi pedestrianstreetcrossactionrecognitioninmonocularfarinfraredsequences |
_version_ |
1721400131404693504 |