Resistive Crossbar-Aware Neural Network Design and Optimization
Recent research in Non-Volatile Memory (NVM) and Processing-in-Memory (PIM) technologies has proposed low energy PIM-based system designs for high-performance neural network inference. Simultaneously, there is a tremendous thrust in neural network architecture research, primarily targeted towards ta...
Main Authors: | , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2020-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9294014/ |
id |
doaj-42d17791149b46a4abd102edeba00b5b |
---|---|
record_format |
Article |
spelling |
doaj-42d17791149b46a4abd102edeba00b5b2021-03-30T04:26:21ZengIEEEIEEE Access2169-35362020-01-01822906622908510.1109/ACCESS.2020.30450719294014Resistive Crossbar-Aware Neural Network Design and OptimizationMuhammad Abdullah Hanif0https://orcid.org/0000-0001-9841-6132Aditya Manglik1Muhammad Shafique2https://orcid.org/0000-0002-2607-8135Faculty of Informatics, Technische Universität Wien (TU Wien), Vienna, AustriaFaculty of Informatics, Technische Universität Wien (TU Wien), Vienna, AustriaDivision of Engineering, New York University Abu Dhabi (NYUAD), Abu Dhabi, United Arab EmiratesRecent research in Non-Volatile Memory (NVM) and Processing-in-Memory (PIM) technologies has proposed low energy PIM-based system designs for high-performance neural network inference. Simultaneously, there is a tremendous thrust in neural network architecture research, primarily targeted towards task-specific accuracy improvements. Despite the enormous potential of a PIM-based compute paradigm, most hardware proposals adopt a one-accelerator-fits-all-networks approach, bleeding performance across all verticals. The overarching goal for this work is to improve the throughput and power efficiency of convolutional neural networks on resistive crossbar-based microarchitectures. To this end, we demonstrate why, how, and where to prune contemporary neural networks for superior exploitation of the crossbar's underlying parallelism model. Further, we present the first crossbar-aware neural network design principles for discovering novel crossbar-amenable network architectures. Our third contribution includes simple yet efficient hardware optimizations to boost energy & area efficiency for modern deep neural networks and ensembles. Finally, we combine these ideas towards our fourth contribution, CrossNet, a novel network architecture family which improves computational efficiency by 19.06× and power efficiency by 4.16× over state-of-the-art designs.https://ieeexplore.ieee.org/document/9294014/Processing-in-memorymemristorcrossbarresistive RAMReRAMneural network |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Muhammad Abdullah Hanif Aditya Manglik Muhammad Shafique |
spellingShingle |
Muhammad Abdullah Hanif Aditya Manglik Muhammad Shafique Resistive Crossbar-Aware Neural Network Design and Optimization IEEE Access Processing-in-memory memristor crossbar resistive RAM ReRAM neural network |
author_facet |
Muhammad Abdullah Hanif Aditya Manglik Muhammad Shafique |
author_sort |
Muhammad Abdullah Hanif |
title |
Resistive Crossbar-Aware Neural Network Design and Optimization |
title_short |
Resistive Crossbar-Aware Neural Network Design and Optimization |
title_full |
Resistive Crossbar-Aware Neural Network Design and Optimization |
title_fullStr |
Resistive Crossbar-Aware Neural Network Design and Optimization |
title_full_unstemmed |
Resistive Crossbar-Aware Neural Network Design and Optimization |
title_sort |
resistive crossbar-aware neural network design and optimization |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2020-01-01 |
description |
Recent research in Non-Volatile Memory (NVM) and Processing-in-Memory (PIM) technologies has proposed low energy PIM-based system designs for high-performance neural network inference. Simultaneously, there is a tremendous thrust in neural network architecture research, primarily targeted towards task-specific accuracy improvements. Despite the enormous potential of a PIM-based compute paradigm, most hardware proposals adopt a one-accelerator-fits-all-networks approach, bleeding performance across all verticals. The overarching goal for this work is to improve the throughput and power efficiency of convolutional neural networks on resistive crossbar-based microarchitectures. To this end, we demonstrate why, how, and where to prune contemporary neural networks for superior exploitation of the crossbar's underlying parallelism model. Further, we present the first crossbar-aware neural network design principles for discovering novel crossbar-amenable network architectures. Our third contribution includes simple yet efficient hardware optimizations to boost energy & area efficiency for modern deep neural networks and ensembles. Finally, we combine these ideas towards our fourth contribution, CrossNet, a novel network architecture family which improves computational efficiency by 19.06× and power efficiency by 4.16× over state-of-the-art designs. |
topic |
Processing-in-memory memristor crossbar resistive RAM ReRAM neural network |
url |
https://ieeexplore.ieee.org/document/9294014/ |
work_keys_str_mv |
AT muhammadabdullahhanif resistivecrossbarawareneuralnetworkdesignandoptimization AT adityamanglik resistivecrossbarawareneuralnetworkdesignandoptimization AT muhammadshafique resistivecrossbarawareneuralnetworkdesignandoptimization |
_version_ |
1724181752589582336 |