Decomposition Methods for Binary and Multi-class Support Vector Machines

• Main Authors: Chih-Wei Hsu, 許智瑋
• Other Authors: Chih-Jen Lin
• Format: Others
• Language: en_US
• Published: 2001
• Online Access: http://ndltd.ncl.edu.tw/handle/18885564730561227819

碩士 === 國立臺灣大學 === 資訊工程學研究所 === 89 === The decomposition method is currently one of the major methods for solving support vector machines (SVM). An important issue of this method is the selection of working sets. In the first part of this thesis through the design of decomposition methods for bound-constrained SVM formulations and from the experimental analysis we propose a simple selection of the working set which leads to faster convergences for difficult cases. Numerical experiments on different types of problems are conducted to demonstrate the viability of the proposed method. The second part of this thesis focuses on decomposition methods for multi-class SVM. As SVM was originally designed for binary classification, how to effectively extend it for multi-class classification is still an on-going research issue. Several methods have been proposed where typically we construct a multi-class classifier by combining several binary classifiers. Some authors also proposed methods that consider all classes of data at once. As it is computationally more expensive on solving multi-class problems, comparisons on these methods using large-scale problems have not been seriously conducted. Especially for methods solving multi-class SVM in one step, a much larger optimization problem is required so up to now experiments are limited to small data sets. In this thesis we give decomposition implementation for two such ``all-together" methods: \cite{VV98a,JW98a} and \cite{KC00a}. We then compare their performance with three methods based on binary classification: ``one-against-all,'' ``one-against-one,'' and DAGSVM \cite{JP00a}. Our experiments indicate that the ``one-against-one'' and DAG methods are more suitable for practical use than the other methods. Results also show that for large problems methods by considering all data at once in general need fewer support vectors.