Statistica Sinica 16(2006), 617-633

MULTI-CATEGORY SUPPORT VECTOR MACHINES,

FEATURE SELECTION AND SOLUTION PATH

Lifeng Wang and Xiaotong Shen

University of Minnesota

Abstract: Support Vector Machines (SVMs) have proven to deliver high performance. However, problems remain with respect to feature selection in multi-category classification. In this article, we propose an algorithm to compute an entire regularization solution path for adaptive feature selection via $L_1$-norm penalized multi-category MSVM (L1MSVM). The advantages of this algorithm are three-fold. First, it permits fast computation for fine tuning, which yields accurate prediction. Second, it greatly reduces the cost of memory. This is especially important in genome classification, where a linear program with tens of thousands of variables has to be solved. Third, it yields a selection order in which the features can be examined sequentially. The performance of the proposed algorithm is examined in simulations and with data.

Key words and phrases: Genome classification, hinge loss, L₁-norm, penalty with, regularization.