Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling

Scene graphs have become one of the hotspots in computer vision research area due to their characteristics of representing the semantic and organizational structure of visual scene content, which facilitates visual comprehension and interpretable inference. However, due to the imbalance of the relat...

Full description

Bibliographic Details
Main Author: TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping
Format: Article
Language:zho
Published: Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press 2021-10-01
Series:Jisuanji kexue yu tansuo
Subjects:
Online Access:http://fcst.ceaj.org/CN/abstract/abstract2919.shtml
id doaj-7a8838c580684aef8c52f25c4c8ec1cf
record_format Article
spelling doaj-7a8838c580684aef8c52f25c4c8ec1cf2021-10-11T08:49:59ZzhoJournal of Computer Engineering and Applications Beijing Co., Ltd., Science PressJisuanji kexue yu tansuo1673-94182021-10-0115101958196810.3778/j.issn.1673-9418.2007007Scene Graph Generation Method Based on External Information Guidance and Residual ScramblingTIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping01. College of Computer Science and Technology, Soochow University, Suzhou, Jiangsu 215006, China 2. Key Laboratory of Symbol Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, ChinaScene graphs have become one of the hotspots in computer vision research area due to their characteristics of representing the semantic and organizational structure of visual scene content, which facilitates visual comprehension and interpretable inference. However, due to the imbalance of the relationship annotation between objects in the visual scene, the existing scene graph generation methods are affected by the bias of the data set. The scene graph data imbalance problem is investigated, and a scene graph generation method based on the combination of external information guidance and residual scrambling (EGRES) is proposed to alleviate the negative impact of data set bias on scene graph generation. This method uses unbiased common sense knowledge in the external knowledge base to standardize the semantic space of the scene graph, alleviate the imbalance of the relational data distribution in the data set, and improve the generalization ability of scene graph generation. The residual scrambling method is used to fuse the visual features and the extracted common sense knowledge to standardize the scene graph generation network. The comparison experiments and ablation experiments on the VG data set prove that the proposed method in this paper can effectively improve the scene graph generation. The comparison experiments for different labels in the data set prove that the proposed method can improve the generation performance of most of the relationship categories, especially in the medium and low frequency relationship categories, which greatly alleviates the imbalance of data labeling and has better generation results than the existing scene graph generation methods.http://fcst.ceaj.org/CN/abstract/abstract2919.shtmldata set biasresidual scramblingexternal knowledge basescene graph generation
collection DOAJ
language zho
format Article
sources DOAJ
author TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping
spellingShingle TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping
Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling
Jisuanji kexue yu tansuo
data set bias
residual scrambling
external knowledge base
scene graph generation
author_facet TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping
author_sort TIAN Xin, JI Yi, GAO Haiyan, LIN Xin, LIU Chunping
title Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling
title_short Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling
title_full Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling
title_fullStr Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling
title_full_unstemmed Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling
title_sort scene graph generation method based on external information guidance and residual scrambling
publisher Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press
series Jisuanji kexue yu tansuo
issn 1673-9418
publishDate 2021-10-01
description Scene graphs have become one of the hotspots in computer vision research area due to their characteristics of representing the semantic and organizational structure of visual scene content, which facilitates visual comprehension and interpretable inference. However, due to the imbalance of the relationship annotation between objects in the visual scene, the existing scene graph generation methods are affected by the bias of the data set. The scene graph data imbalance problem is investigated, and a scene graph generation method based on the combination of external information guidance and residual scrambling (EGRES) is proposed to alleviate the negative impact of data set bias on scene graph generation. This method uses unbiased common sense knowledge in the external knowledge base to standardize the semantic space of the scene graph, alleviate the imbalance of the relational data distribution in the data set, and improve the generalization ability of scene graph generation. The residual scrambling method is used to fuse the visual features and the extracted common sense knowledge to standardize the scene graph generation network. The comparison experiments and ablation experiments on the VG data set prove that the proposed method in this paper can effectively improve the scene graph generation. The comparison experiments for different labels in the data set prove that the proposed method can improve the generation performance of most of the relationship categories, especially in the medium and low frequency relationship categories, which greatly alleviates the imbalance of data labeling and has better generation results than the existing scene graph generation methods.
topic data set bias
residual scrambling
external knowledge base
scene graph generation
url http://fcst.ceaj.org/CN/abstract/abstract2919.shtml
work_keys_str_mv AT tianxinjiyigaohaiyanlinxinliuchunping scenegraphgenerationmethodbasedonexternalinformationguidanceandresidualscrambling
_version_ 1716827946629464064