Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies, Opportunities, and Challenges
Deep learning (DL) has seen great success in the computer vision (CV) field, and related techniques have been used in security, healthcare, remote sensing, and many other areas. As a parallel development, visual data has become universal in daily life, easily generated by ubiquitous low-cost cameras...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2021-01-01
|
Series: | IEEE Open Journal of the Communications Society |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9305715/ |
id |
doaj-e561fb0e65304838aee9fad201957c74 |
---|---|
record_format |
Article |
spelling |
doaj-e561fb0e65304838aee9fad201957c742021-03-29T18:58:10ZengIEEEIEEE Open Journal of the Communications Society2644-125X2021-01-01213214310.1109/OJCOMS.2020.30426309305715Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies, Opportunities, and ChallengesYu Tian0https://orcid.org/0000-0003-3394-3219Gaofeng Pan1https://orcid.org/0000-0003-1008-5717Mohamed-Slim Alouini2https://orcid.org/0000-0003-4827-1793Computer, Electrical and Mathematical Science and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi ArabiaComputer, Electrical and Mathematical Science and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi ArabiaComputer, Electrical and Mathematical Science and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi ArabiaDeep learning (DL) has seen great success in the computer vision (CV) field, and related techniques have been used in security, healthcare, remote sensing, and many other areas. As a parallel development, visual data has become universal in daily life, easily generated by ubiquitous low-cost cameras. Therefore, exploring DL-based CV may yield useful information about objects, such as their number, locations, distribution, motion, etc. Intuitively, DL-based CV can also facilitate and improve the designs of wireless communications, especially in dynamic network scenarios. However, so far, such work is rare in the literature. The primary purpose of this article, then, is to introduce ideas about applying DL-based CV in wireless communications to bring some novel degrees of freedom to both theoretical research and engineering applications. To illustrate how DL-based CV can be applied in wireless communications, an example of using a DL-based CV with a millimeter-wave (mmWave) system is given to realize optimal mmWave multiple-input and multiple-output (MIMO) beamforming in mobile scenarios. In this example, we propose a framework to predict future beam indices from previously observed beam indices and images of street views using ResNet, 3-dimensional ResNext, and a long short-term memory network. The experimental results show that our frameworks achieve much higher accuracy than the baseline method, and that visual data can significantly improve the performance of the MIMO beamforming system. Finally, we discuss the opportunities and challenges of applying DL-based CV in wireless communications.https://ieeexplore.ieee.org/document/9305715/Computer visiondeep learningmultiple-input and multiple-outputbeamformingbeam trackinglong short-term memory |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Yu Tian Gaofeng Pan Mohamed-Slim Alouini |
spellingShingle |
Yu Tian Gaofeng Pan Mohamed-Slim Alouini Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies, Opportunities, and Challenges IEEE Open Journal of the Communications Society Computer vision deep learning multiple-input and multiple-output beamforming beam tracking long short-term memory |
author_facet |
Yu Tian Gaofeng Pan Mohamed-Slim Alouini |
author_sort |
Yu Tian |
title |
Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies, Opportunities, and Challenges |
title_short |
Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies, Opportunities, and Challenges |
title_full |
Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies, Opportunities, and Challenges |
title_fullStr |
Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies, Opportunities, and Challenges |
title_full_unstemmed |
Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies, Opportunities, and Challenges |
title_sort |
applying deep-learning-based computer vision to wireless communications: methodologies, opportunities, and challenges |
publisher |
IEEE |
series |
IEEE Open Journal of the Communications Society |
issn |
2644-125X |
publishDate |
2021-01-01 |
description |
Deep learning (DL) has seen great success in the computer vision (CV) field, and related techniques have been used in security, healthcare, remote sensing, and many other areas. As a parallel development, visual data has become universal in daily life, easily generated by ubiquitous low-cost cameras. Therefore, exploring DL-based CV may yield useful information about objects, such as their number, locations, distribution, motion, etc. Intuitively, DL-based CV can also facilitate and improve the designs of wireless communications, especially in dynamic network scenarios. However, so far, such work is rare in the literature. The primary purpose of this article, then, is to introduce ideas about applying DL-based CV in wireless communications to bring some novel degrees of freedom to both theoretical research and engineering applications. To illustrate how DL-based CV can be applied in wireless communications, an example of using a DL-based CV with a millimeter-wave (mmWave) system is given to realize optimal mmWave multiple-input and multiple-output (MIMO) beamforming in mobile scenarios. In this example, we propose a framework to predict future beam indices from previously observed beam indices and images of street views using ResNet, 3-dimensional ResNext, and a long short-term memory network. The experimental results show that our frameworks achieve much higher accuracy than the baseline method, and that visual data can significantly improve the performance of the MIMO beamforming system. Finally, we discuss the opportunities and challenges of applying DL-based CV in wireless communications. |
topic |
Computer vision deep learning multiple-input and multiple-output beamforming beam tracking long short-term memory |
url |
https://ieeexplore.ieee.org/document/9305715/ |
work_keys_str_mv |
AT yutian applyingdeeplearningbasedcomputervisiontowirelesscommunicationsmethodologiesopportunitiesandchallenges AT gaofengpan applyingdeeplearningbasedcomputervisiontowirelesscommunicationsmethodologiesopportunitiesandchallenges AT mohamedslimalouini applyingdeeplearningbasedcomputervisiontowirelesscommunicationsmethodologiesopportunitiesandchallenges |
_version_ |
1724196206489370624 |