CMBF: Cross-Modal-Based Fusion Recommendation Algorithm

• Main Authors: Xi Chen, Yangsiyi Lu, Yuehai Wang, Jianyi Yang
• Format: Article
• Language: English
• Published: MDPI AG 2021-08-01
• Series: Sensors
• Subjects: recommendation systems; multi-modal algorithm; cross-modal fusion; attention mechanism
• Online Access: https://www.mdpi.com/1424-8220/21/16/5275

A recommendation system is often used to recommend items that may be of interest to users. One of the main challenges is that the scarcity of actual interaction data between users and items restricts the performance of recommendation systems. To solve this problem, multi-modal technologies have been used for expanding available information. However, the existing multi-modal recommendation algorithms all extract the feature of single modality and simply splice the features of different modalities to predict the recommendation results. This fusion method can not completely mine the relevance of multi-modal features and lose the relationship between different modalities, which affects the prediction results. In this paper, we propose a Cross-Modal-Based Fusion Recommendation Algorithm (CMBF) that can capture both the single-modal features and the cross-modal features. Our algorithm uses a novel cross-modal fusion method to fuse the multi-modal features completely and learn the cross information between different modalities. We evaluate our algorithm on two datasets, MovieLens and Amazon. Experiments show that our method has achieved the best performance compared to other recommendation algorithms. We also design ablation study to prove that our cross-modal fusion method improves the prediction results.