Statistica Sinica 18(2008), 379-398

THE $\mbox{\boldmath $F_{\infty}$}$-NORM SUPPORT VECTOR MACHINE

Hui Zou and Ming Yuan

University of Minnesota and Georgia Institute of Technology

Abstract: In this paper we propose a new support vector machine (SVM), the $F_{\infty}$-norm SVM, to perform automatic factor selection in classification. The $F_{\infty}$-norm SVM methodology is motivated by the feature selection problem in cases where the input features are generated by factors, and the model is best interpreted in terms of significant factors. This type of problem arises naturally when a set of dummy variables is used to represent a categorical factor and/or a set of basis functions of a continuous variable is included in the predictor set. In problems without such obvious group information, we propose to first create groups among features by clustering, and then apply the $F_{\infty}$-norm SVM. We show that the $F_{\infty}$-norm SVM is equivalent to a linear programming problem and can be efficiently solved using standard techniques. Analysis on simulated and real data shows that the $F_{\infty}$-norm SVM enjoys competitive performance when compared with the 1-norm and 2-norm SVMs.

Key words and phrases: F∞ penalty, factor selection; feature selection, linear programming, L∞ penalty, support vector machine.