Filter Pruning Without Damaging Networks Capacity
Due to its over-parameterized design, the deep convolutional neural networks lead to a huge amount of parameters and high computational cost, making it difficult to deploy on some devices with limited computational resources in reality. In this paper, we propose a method of filter pruning without da...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2020-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9091183/ |
id |
doaj-a9db86aa47d8482f9b79753d5baaf0ec |
---|---|
record_format |
Article |
spelling |
doaj-a9db86aa47d8482f9b79753d5baaf0ec2021-03-30T02:37:41ZengIEEEIEEE Access2169-35362020-01-018909249093010.1109/ACCESS.2020.29939329091183Filter Pruning Without Damaging Networks CapacityYuding Zuo0https://orcid.org/0000-0003-3550-0737Bo Chen1https://orcid.org/0000-0001-6621-2215Te Shi2https://orcid.org/0000-0002-9906-1908Mengfan Sun3https://orcid.org/0000-0002-2195-1521School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaSchool of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaSchool of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaSchool of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaDue to its over-parameterized design, the deep convolutional neural networks lead to a huge amount of parameters and high computational cost, making it difficult to deploy on some devices with limited computational resources in reality. In this paper, we propose a method of filter pruning without damaging networks capacity to accelerate and compress deep convolutional neural networks. Differently from some existing filter pruning methods, we pay more attention to the damage by filter pruning to model capacity. In order to restore the original model capacity, we generate new feature maps on the basis of the remaining feature maps with lighter structure after pruning the redundant filters that are similar with the others. Experimental results on CIFAR10 and CIFAR100 benchmarks demonstrate the effectiveness of our method. Especially, our method reduces more than 49% FLOPs for VGGNet-16 on CIFAR10 with only 0.07% relative accuracy drop. The relative accuracy has even been increased by 0.13% with reducing more than 24% FLOPs. Moreover, our method accelerates ResNet-110 on CIFAR10 by 22.1% with 0.41% accuracy improvement, which exceeds the previous methods.https://ieeexplore.ieee.org/document/9091183/Convolutional neural networksfilter pruningfilter similaritymodel capacitynetworks compression and acceleration |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Yuding Zuo Bo Chen Te Shi Mengfan Sun |
spellingShingle |
Yuding Zuo Bo Chen Te Shi Mengfan Sun Filter Pruning Without Damaging Networks Capacity IEEE Access Convolutional neural networks filter pruning filter similarity model capacity networks compression and acceleration |
author_facet |
Yuding Zuo Bo Chen Te Shi Mengfan Sun |
author_sort |
Yuding Zuo |
title |
Filter Pruning Without Damaging Networks Capacity |
title_short |
Filter Pruning Without Damaging Networks Capacity |
title_full |
Filter Pruning Without Damaging Networks Capacity |
title_fullStr |
Filter Pruning Without Damaging Networks Capacity |
title_full_unstemmed |
Filter Pruning Without Damaging Networks Capacity |
title_sort |
filter pruning without damaging networks capacity |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2020-01-01 |
description |
Due to its over-parameterized design, the deep convolutional neural networks lead to a huge amount of parameters and high computational cost, making it difficult to deploy on some devices with limited computational resources in reality. In this paper, we propose a method of filter pruning without damaging networks capacity to accelerate and compress deep convolutional neural networks. Differently from some existing filter pruning methods, we pay more attention to the damage by filter pruning to model capacity. In order to restore the original model capacity, we generate new feature maps on the basis of the remaining feature maps with lighter structure after pruning the redundant filters that are similar with the others. Experimental results on CIFAR10 and CIFAR100 benchmarks demonstrate the effectiveness of our method. Especially, our method reduces more than 49% FLOPs for VGGNet-16 on CIFAR10 with only 0.07% relative accuracy drop. The relative accuracy has even been increased by 0.13% with reducing more than 24% FLOPs. Moreover, our method accelerates ResNet-110 on CIFAR10 by 22.1% with 0.41% accuracy improvement, which exceeds the previous methods. |
topic |
Convolutional neural networks filter pruning filter similarity model capacity networks compression and acceleration |
url |
https://ieeexplore.ieee.org/document/9091183/ |
work_keys_str_mv |
AT yudingzuo filterpruningwithoutdamagingnetworkscapacity AT bochen filterpruningwithoutdamagingnetworkscapacity AT teshi filterpruningwithoutdamagingnetworkscapacity AT mengfansun filterpruningwithoutdamagingnetworkscapacity |
_version_ |
1724184876635127808 |