A Projection Multi-objective SVM Method for Multi-class Classification

Support Vector Machines (SVMs) have become a very popular technique in the machine learning field for classification problems. They were originally proposed for classifications of two classes. For multi-class classifications, various single-objective models have been proposed mostly based on two families of methods: an all-together approach and a combination of binary classifiers. However, most of these single-objective models consider neither the different costs of different misclassifications nor the user’s preferences.
To overcome these drawbacks, multi-objective models have been proposed. By solving a large second-order cone program (SOCP), these multi-objective models give us weakly Pareto-optimal solutions. The need of solving large SOCPs makes these multi-objective models impractical when we have a large amount of classes to classify. In this paper, we propose a multi-objective technique that we denominate Projected Multi-objective SVM (PM), which works in a higher dimensional space than the object space. For PM, we can characterize these Pareto-optimal solutions. And for classifications with large numbers of
classes, with PM, we can significantly alleviate the computational bottleneck. From our experimental results, and compared with the single-objective multi-class SVMs (based on an all-together method, one-against-all method and one-against-one method), PM obtains comparable values for the training classification accuracies, testing classification accuracies and training time. Compared to other published multi-objective multi-class SVMs, PM gives higher Pareto-compliant indicator values while requiring less computation time.