Statistica Sinica 28 (2018), 449-469
Abstract: In high-dimensional data analysis, we propose a sequential model averaging (SMA) method to make accurate and stable predictions. Specifically, we introduce a hybrid approach that combines a sequential screening process with a model averaging algorithm, where the weight of each model is determined by its Bayesian information (BIC) score (Schwarz (1978); Chen and Chen (2008)). The sequential technique makes SMA computationally feasible with high-dimensional data, because the averaging process assures the prediction's accuracy and stability. Results show that SMA not only yields a good model, but also mitigates overfitting. We demonstrate that SMA provides consistent estimators for the regression coefficients and yields reliable predictions under mild conditions. Simulations and empirical examples are presented to illustrate the usefulness of the proposed method.
Key words and phrases: Forward regression, sequential model averaging, sequential screening, univariate model averaging.