An interactive instance segmentation system with multi‐resolution convolutional neural networks
Abstract In this paper, a fast interactive instance segmentation (IIS) system is proposed and it is composed of an effective heatmap generator, a multi‐resolution network (MRNet), and an adaptive threshold refiner to promptly and precisely predict the masks of the objects. The proposed heatmap gener...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Wiley
2021-03-01
|
Series: | IET Computer Vision |
Online Access: | https://doi.org/10.1049/cvi2.12016 |
id |
doaj-88c3901986594721985891f497ed1c67 |
---|---|
record_format |
Article |
spelling |
doaj-88c3901986594721985891f497ed1c672021-09-30T04:50:43ZengWileyIET Computer Vision1751-96321751-96402021-03-011529910910.1049/cvi2.12016An interactive instance segmentation system with multi‐resolution convolutional neural networksPo‐Wei Sung0Wei‐Jong Yang1Jar‐Ferr Yang2Din‐Yuan Chan3Department of Electrical Engineering Institute of Computer and Communication Engineering National Cheng Kung University Tainan TaiwanDepartment of Electrical Engineering Institute of Computer and Communication Engineering National Cheng Kung University Tainan TaiwanDepartment of Electrical Engineering Institute of Computer and Communication Engineering National Cheng Kung University Tainan TaiwanDepartment of Computer Science and Information Engineering National Chia‐Yi University Chia‐Yi TaiwanAbstract In this paper, a fast interactive instance segmentation (IIS) system is proposed and it is composed of an effective heatmap generator, a multi‐resolution network (MRNet), and an adaptive threshold refiner to promptly and precisely predict the masks of the objects. The proposed heatmap generator after interaction clicks can help the MRNet to successfully learn the sensitive features for better prediction. Based on convolutional neural network models, the proposed MRNet backbone produces multiple features across multiple resolutions and can intrinsically predict the sharp contour of the object. After the probabilistic prediction achieved by the MRNet, the Otsu's threshold refiner is proposed to further remove some uncertain pixels in the predicted mask. Experimental results demonstrate that the proposed IIS system can promptly predict sharp masks of the targeted objects with mIoU of 89.1% in PASCAL VOC 2012 [1] validation set. Compared to other existing interactive methods, the proposed system can effectively predict the segmentation mask with higher accuracy and less interaction efforts.https://doi.org/10.1049/cvi2.12016 |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Po‐Wei Sung Wei‐Jong Yang Jar‐Ferr Yang Din‐Yuan Chan |
spellingShingle |
Po‐Wei Sung Wei‐Jong Yang Jar‐Ferr Yang Din‐Yuan Chan An interactive instance segmentation system with multi‐resolution convolutional neural networks IET Computer Vision |
author_facet |
Po‐Wei Sung Wei‐Jong Yang Jar‐Ferr Yang Din‐Yuan Chan |
author_sort |
Po‐Wei Sung |
title |
An interactive instance segmentation system with multi‐resolution convolutional neural networks |
title_short |
An interactive instance segmentation system with multi‐resolution convolutional neural networks |
title_full |
An interactive instance segmentation system with multi‐resolution convolutional neural networks |
title_fullStr |
An interactive instance segmentation system with multi‐resolution convolutional neural networks |
title_full_unstemmed |
An interactive instance segmentation system with multi‐resolution convolutional neural networks |
title_sort |
interactive instance segmentation system with multi‐resolution convolutional neural networks |
publisher |
Wiley |
series |
IET Computer Vision |
issn |
1751-9632 1751-9640 |
publishDate |
2021-03-01 |
description |
Abstract In this paper, a fast interactive instance segmentation (IIS) system is proposed and it is composed of an effective heatmap generator, a multi‐resolution network (MRNet), and an adaptive threshold refiner to promptly and precisely predict the masks of the objects. The proposed heatmap generator after interaction clicks can help the MRNet to successfully learn the sensitive features for better prediction. Based on convolutional neural network models, the proposed MRNet backbone produces multiple features across multiple resolutions and can intrinsically predict the sharp contour of the object. After the probabilistic prediction achieved by the MRNet, the Otsu's threshold refiner is proposed to further remove some uncertain pixels in the predicted mask. Experimental results demonstrate that the proposed IIS system can promptly predict sharp masks of the targeted objects with mIoU of 89.1% in PASCAL VOC 2012 [1] validation set. Compared to other existing interactive methods, the proposed system can effectively predict the segmentation mask with higher accuracy and less interaction efforts. |
url |
https://doi.org/10.1049/cvi2.12016 |
work_keys_str_mv |
AT poweisung aninteractiveinstancesegmentationsystemwithmultiresolutionconvolutionalneuralnetworks AT weijongyang aninteractiveinstancesegmentationsystemwithmultiresolutionconvolutionalneuralnetworks AT jarferryang aninteractiveinstancesegmentationsystemwithmultiresolutionconvolutionalneuralnetworks AT dinyuanchan aninteractiveinstancesegmentationsystemwithmultiresolutionconvolutionalneuralnetworks AT poweisung interactiveinstancesegmentationsystemwithmultiresolutionconvolutionalneuralnetworks AT weijongyang interactiveinstancesegmentationsystemwithmultiresolutionconvolutionalneuralnetworks AT jarferryang interactiveinstancesegmentationsystemwithmultiresolutionconvolutionalneuralnetworks AT dinyuanchan interactiveinstancesegmentationsystemwithmultiresolutionconvolutionalneuralnetworks |
_version_ |
1716863945565798400 |