Abstract: We investigate two important properties of M-estimators, namely, robustness and tractability, in a linear regression setting, when the observations are contaminated by some arbitrary outliers. Specifically, robustness is the statistical property that the estimator should always be close to the true underlying parameters, regardless of the distribution of the outliers, and tractability refers to the computational property that the estimator can be computed efficiently, even if the objective function of the M-estimator is nonconvex. In this article, by examining the empirical risk, we show that under some sufficient conditions, many M-estimators enjoy nice robustness and tractability properties simultaneously when the percentage of outliers is small. We extend our analysis to the high-imensional setting, where the number of parameters is greater than the number of samples, p ≫ 𝓃, and prove that when the proportion of outliers is small, the penalized M-estimators with the L₁ penalty enjoy robustness and tractability simultaneously. Our research provides an analytic approach to determine the effects of outliers and tuning parameters on the robustness and tractability of some families of M-estimators. Simulations and case studies are presented to illustrate the usefulness of our theoretical results for M-estimators under Welsch's exponential squared loss and Tukey's bisquare loss.

Key words and phrases: Computational tractability, gross error, high-dimensionality, nonconvexity, robust regression, sparsity.