結合局部特徵序列的影片背景音樂推薦機制

• Main Authors: 林鼎崴, Lin, Ting Wei
• Language: 中文
• Published: 國立政治大學
• Subjects: 影片自動配樂; 資料探勘; 關聯模型; 背景音樂推薦
• Online Access: http://thesis.lib.nccu.edu.tw/cgi-bin/cdrfb3/gsweb.cgi?o=dstdcdr&i=sid=%22G0103753008%22.

隨著手持裝置的普及與社群網路的興起，大眾可以隨時拍攝影片並且上傳至網路上與他人分享。但是一般使用者產生的影片若少了配樂，將失色許多。除了原本影片帶給人們的視覺觀感之外，配樂可以帶給人們聽覺的觀感，進而使得人們可以更容易了解影片的情感，也可以讓人們更能夠融入在影片中。背景音樂推薦的研究主要有兩大種做法，Emotion-mediated Approach與Correlation-based Approach。我們使用Correlational-based Approach的方法，利用Correlation Modeling找出影片特徵值與音樂特徵值之間的關係。但是由於目前Correlation-based Approach的研究只有考慮到全域特徵，因此在此論文中，我們提出了區域特徵。區域特徵利用時間序列表達影片細部的變化，並且將區域特徵與全域特徵結合至Correlation Modeling中，透過 MLSA、CFA、CCA、KCCA、DCCA、PLS、PLSR演算法找出其中的關係並且產生背景音樂推薦的Ranking List，實驗部份比較了各個演算法在背景音樂推薦上的準確率，並且觀察Global Features與Local Features之間的準確率。 === Background music plays an important role in making user-generated video more colorful and attractive. One of current research on automatic background music recommendation is the correlation-based approach in which the correlation model between visual and music features is discovered from training data and is utilized to recommend background music for query video. Because the existing correlation-based approaches consider global features only, in this work we proposed to integrate the temporal sequence of local features along with global features into the correlation modeling process. The local features are derived from segmented audiovisual clips and can represent the local variation of features. Then the temporal sequence of local features is transformed and incorporated into correlation modeling process. Cross-Modal Factor Analysis along with Multiple-type Latent Semantic Analysis, Canonical Correlation Analysis, Kernel Canonical Correlation Analysis, Deep Canonical Correlation Analysis, Partial Least Square and Partial Least Square Regression, are investigated for correlation modeling which recommends background music in ranking order. In the experiments, we first compare the results of only global features, only local Features and incorporating global and local Features among each algorithm. Then second compare the results of different clip numbers and Fourier coefficients.