MultEYE: Monitoring System for Real-Time Vehicle Detection, Tracking and Speed Estimation from UAV Imagery on Edge-Computing Platforms
We present MultEYE, a traffic monitoring system that can detect, track, and estimate the velocity of vehicles in a sequence of aerial images. The presented solution has been optimized to execute these tasks in real-time on an embedded computer installed on an Unmanned Aerial Vehicle (UAV). In order...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2021-02-01
|
Series: | Remote Sensing |
Subjects: | |
Online Access: | https://www.mdpi.com/2072-4292/13/4/573 |
id |
doaj-92a6ec5e94f541b0b0a6f72beecf08fd |
---|---|
record_format |
Article |
spelling |
doaj-92a6ec5e94f541b0b0a6f72beecf08fd2021-02-06T00:06:24ZengMDPI AGRemote Sensing2072-42922021-02-011357357310.3390/rs13040573MultEYE: Monitoring System for Real-Time Vehicle Detection, Tracking and Speed Estimation from UAV Imagery on Edge-Computing PlatformsNavaneeth Balamuralidhar0Sofia Tilon1Francesco Nex2Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, 7514 AE Enschede, The NetherlandsFaculty of Geo-Information Science and Earth Observation (ITC), University of Twente, 7514 AE Enschede, The NetherlandsFaculty of Geo-Information Science and Earth Observation (ITC), University of Twente, 7514 AE Enschede, The NetherlandsWe present MultEYE, a traffic monitoring system that can detect, track, and estimate the velocity of vehicles in a sequence of aerial images. The presented solution has been optimized to execute these tasks in real-time on an embedded computer installed on an Unmanned Aerial Vehicle (UAV). In order to overcome the limitation of existing object detection architectures related to accuracy and computational overhead, a multi-task learning methodology was employed by adding a segmentation head to an object detector backbone resulting in the MultEYE object detection architecture. On a custom dataset, it achieved 4.8% higher mean Average Precision (mAP) score, while being 91.4% faster than the state-of-the-art model and while being able to generalize to different real-world traffic scenes. Dedicated object tracking and speed estimation algorithms have been then optimized to track reliably objects from an UAV with limited computational effort. Different strategies to combine object detection, tracking, and speed estimation are discussed, too. From our experiments, the optimized detector runs at an average frame-rate of up to 29 frames per second (FPS) on frame resolution 512 × 320 on a Nvidia Xavier NX board, while the optimally combined detector, tracker and speed estimator pipeline achieves speeds of up to 33 FPS on an image of resolution 3072 × 1728. To our knowledge, the MultEYE system is one of the first traffic monitoring systems that was specifically designed and optimized for an UAV platform under real-world constraints.https://www.mdpi.com/2072-4292/13/4/573multi-task learningtraffic monitoringvehicle detectionvehicle trackingUnmanned Aerial Vehiclesobject detection |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Navaneeth Balamuralidhar Sofia Tilon Francesco Nex |
spellingShingle |
Navaneeth Balamuralidhar Sofia Tilon Francesco Nex MultEYE: Monitoring System for Real-Time Vehicle Detection, Tracking and Speed Estimation from UAV Imagery on Edge-Computing Platforms Remote Sensing multi-task learning traffic monitoring vehicle detection vehicle tracking Unmanned Aerial Vehicles object detection |
author_facet |
Navaneeth Balamuralidhar Sofia Tilon Francesco Nex |
author_sort |
Navaneeth Balamuralidhar |
title |
MultEYE: Monitoring System for Real-Time Vehicle Detection, Tracking and Speed Estimation from UAV Imagery on Edge-Computing Platforms |
title_short |
MultEYE: Monitoring System for Real-Time Vehicle Detection, Tracking and Speed Estimation from UAV Imagery on Edge-Computing Platforms |
title_full |
MultEYE: Monitoring System for Real-Time Vehicle Detection, Tracking and Speed Estimation from UAV Imagery on Edge-Computing Platforms |
title_fullStr |
MultEYE: Monitoring System for Real-Time Vehicle Detection, Tracking and Speed Estimation from UAV Imagery on Edge-Computing Platforms |
title_full_unstemmed |
MultEYE: Monitoring System for Real-Time Vehicle Detection, Tracking and Speed Estimation from UAV Imagery on Edge-Computing Platforms |
title_sort |
multeye: monitoring system for real-time vehicle detection, tracking and speed estimation from uav imagery on edge-computing platforms |
publisher |
MDPI AG |
series |
Remote Sensing |
issn |
2072-4292 |
publishDate |
2021-02-01 |
description |
We present MultEYE, a traffic monitoring system that can detect, track, and estimate the velocity of vehicles in a sequence of aerial images. The presented solution has been optimized to execute these tasks in real-time on an embedded computer installed on an Unmanned Aerial Vehicle (UAV). In order to overcome the limitation of existing object detection architectures related to accuracy and computational overhead, a multi-task learning methodology was employed by adding a segmentation head to an object detector backbone resulting in the MultEYE object detection architecture. On a custom dataset, it achieved 4.8% higher mean Average Precision (mAP) score, while being 91.4% faster than the state-of-the-art model and while being able to generalize to different real-world traffic scenes. Dedicated object tracking and speed estimation algorithms have been then optimized to track reliably objects from an UAV with limited computational effort. Different strategies to combine object detection, tracking, and speed estimation are discussed, too. From our experiments, the optimized detector runs at an average frame-rate of up to 29 frames per second (FPS) on frame resolution 512 × 320 on a Nvidia Xavier NX board, while the optimally combined detector, tracker and speed estimator pipeline achieves speeds of up to 33 FPS on an image of resolution 3072 × 1728. To our knowledge, the MultEYE system is one of the first traffic monitoring systems that was specifically designed and optimized for an UAV platform under real-world constraints. |
topic |
multi-task learning traffic monitoring vehicle detection vehicle tracking Unmanned Aerial Vehicles object detection |
url |
https://www.mdpi.com/2072-4292/13/4/573 |
work_keys_str_mv |
AT navaneethbalamuralidhar multeyemonitoringsystemforrealtimevehicledetectiontrackingandspeedestimationfromuavimageryonedgecomputingplatforms AT sofiatilon multeyemonitoringsystemforrealtimevehicledetectiontrackingandspeedestimationfromuavimageryonedgecomputingplatforms AT francesconex multeyemonitoringsystemforrealtimevehicledetectiontrackingandspeedestimationfromuavimageryonedgecomputingplatforms |
_version_ |
1724282633422110720 |