Statistica Sinica 27 (2017), 1243-1264

SPARSE AND ROBUST LINEAR REGRESSION:

AN OPTIMIZATION ALGORITHM AND

ITS STATISTICAL PROPERTIES

Shota Katayama and Hironori Fujisawa

Tokyo Institute of Technology and The Institute of Statistical Mathematics

Abstract: This paper studies sparse linear regression analysis with
outliers in the responses. A parameter vector for modeling outliers is added to the
standard linear
regression model and then the sparse estimation problem for both coefficients and
outliers is considered. 𝓁_{1} penalty is imposed for the coefficients, while various penalties including redescending type penalties are for the outliers. To
solve the sparse estimation problem, we introduce an optimization algorithm. Under some
conditions, we show the algorithmic and statistical convergence property for the
coefficients obtained by the algorithm. Moreover, it is shown that the algorithm
can recover the true support of the coefficients with probability going to one.

Key words and phrases: Algorithmic and statistical convergence, robust estimation, sparse linear regression, support recovery.