Abstract: We test the number of spikes in a generalized spiked covariance matrix, the spiked eigenvalues of which may be much larger or smaller than the nonspiked ones. For high-dimensional problems, we first propose a general test statistic and derive its central limit theorem using random matrix theory without a Gaussian population constraint. We then apply the result to estimate the noise variance and test the equality of the smallest roots in generalized spiked models. The results of our simulation studies show that the proposed test method is sized correctly, and the power outcomes demonstrate the robustness of our statistic to deviations from a Gaussian population. Moreover, our estimator of the noise variance results in much smaller mean absolute errors and mean squared errors than those of existing methods. In contrast to other methods, we eliminate the strict conditions of a diagonal or a block-wise diagonal form of the population covariance matrix, and extend the work to a wider range, without the assumption of normality. Thus, the proposed method is highly suitable for real problems.

Key words and phrases: Central limit theorem, generalized spiked model, high-dimensional covariance matrix, testing the spikes.