Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling
Scene graphs have become one of the hotspots in computer vision research area due to their characteristics of representing the semantic and organizational structure of visual scene content, which facilitates visual comprehension and interpretable inference. However, due to the imbalance of the relat...
Main Author: | |
---|---|
Format: | Article |
Language: | zho |
Published: |
Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press
2021-10-01
|
Series: | Jisuanji kexue yu tansuo |
Subjects: | |
Online Access: | http://fcst.ceaj.org/CN/abstract/abstract2919.shtml |
id |
doaj-7a8838c580684aef8c52f25c4c8ec1cf |
---|---|
record_format |
Article |
spelling |
doaj-7a8838c580684aef8c52f25c4c8ec1cf2021-10-11T08:49:59ZzhoJournal of Computer Engineering and Applications Beijing Co., Ltd., Science PressJisuanji kexue yu tansuo1673-94182021-10-0115101958196810.3778/j.issn.1673-9418.2007007Scene Graph Generation Method Based on External Information Guidance and Residual ScramblingTIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping01. College of Computer Science and Technology, Soochow University, Suzhou, Jiangsu 215006, China 2. Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, ChinaScene graphs have become one of the hotspots in computer vision research area due to their characteristics of representing the semantic and organizational structure of visual scene content, which facilitates visual comprehension and interpretable inference. However, due to the imbalance of the relationship annotation between objects in the visual scene, the existing scene graph generation methods are affected by the bias of the data set. The scene graph data imbalance problem is investigated, and a scene graph generation method based on the combination of external information guidance and residual scrambling (EGRES) is proposed to alleviate the negative impact of data set bias on scene graph generation. This method uses unbiased common sense knowledge in the external knowledge base to standardize the semantic space of the scene graph, alleviate the imbalance of the relational data distribution in the data set, and improve the generalization ability of scene graph generation. The residual scrambling method is used to fuse the visual features and the extracted common sense knowledge to standardize the scene graph generation network. The comparison experiments and ablation experiments on the VG data set prove that the proposed method in this paper can effectively improve the scene graph generation. The comparison experiments for different labels in the data set prove that the proposed method can improve the generation performance of most of the relationship categories, especially in the medium and low frequency relationship categories, which greatly alleviates the imbalance of data labeling and has better generation results than the existing scene graph generation methods.http://fcst.ceaj.org/CN/abstract/abstract2919.shtmldata set biasresidual scramblingexternal knowledge basescene graph generation |
collection |
DOAJ |
language |
zho |
format |
Article |
sources |
DOAJ |
author |
TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping |
spellingShingle |
TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling Jisuanji kexue yu tansuo data set bias residual scrambling external knowledge base scene graph generation |
author_facet |
TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping |
author_sort |
TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping |
title |
Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling |
title_short |
Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling |
title_full |
Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling |
title_fullStr |
Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling |
title_full_unstemmed |
Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling |
title_sort |
scene graph generation method based on external information guidance and residual scrambling |
publisher |
Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press |
series |
Jisuanji kexue yu tansuo |
issn |
1673-9418 |
publishDate |
2021-10-01 |
description |
Scene graphs have become one of the hotspots in computer vision research area due to their characteristics of representing the semantic and organizational structure of visual scene content, which facilitates visual comprehension and interpretable inference. However, due to the imbalance of the relationship annotation between objects in the visual scene, the existing scene graph generation methods are affected by the bias of the data set. The scene graph data imbalance problem is investigated, and a scene graph generation method based on the combination of external information guidance and residual scrambling (EGRES) is proposed to alleviate the negative impact of data set bias on scene graph generation. This method uses unbiased common sense knowledge in the external knowledge base to standardize the semantic space of the scene graph, alleviate the imbalance of the relational data distribution in the data set, and improve the generalization ability of scene graph generation. The residual scrambling method is used to fuse the visual features and the extracted common sense knowledge to standardize the scene graph generation network. The comparison experiments and ablation experiments on the VG data set prove that the proposed method in this paper can effectively improve the scene graph generation. The comparison experiments for different labels in the data set prove that the proposed method can improve the generation performance of most of the relationship categories, especially in the medium and low frequency relationship categories, which greatly alleviates the imbalance of data labeling and has better generation results than the existing scene graph generation methods. |
topic |
data set bias residual scrambling external knowledge base scene graph generation |
url |
http://fcst.ceaj.org/CN/abstract/abstract2919.shtml |
work_keys_str_mv |
AT tianxinjiyigaohaiyanlinxinliuchunping scenegraphgenerationmethodbasedonexternalinformationguidanceandresidualscrambling |
_version_ |
1716827946629464064 |