Statistica Sinica 13(2003), 351-365

TESTS FOR HOMOGENEITY IN NORMAL MIXTURES IN

THE PRESENCE OF A STRUCTURAL PARAMETER

Hanfeng Chen and Jiahua Chen

Bowling Green State University and University of Waterloo

Abstract: Often a question arises as to whether observed data are a sample from a homogeneous population or from a heterogeneous population. If in particular, one wants to test for a single normal distribution versus a mixture of two normal distributions, classic asymptotic results do not apply since the model does not satisfy regularity conditions. This paper investigates the large sample behavior of the likelihood ratio statistic for testing homogeneity in the normal mixture in location parameters with an unknown structural parameter. It is proved that the asymptotic null distribution of the likelihood ratio statistic is the maximum of a $\chi^2_2$-variable and the supremum of the square of a truncated Gaussian process with mean 0 and variance 1. This result exposes the unusual large sample behavior of the likelihood function under the null distribution. The correlation structure of the process involved in the limiting distribution is presented explicitly. From the large sample study, it is also found that even though the structural parameter is not part of the mixing distribution, the convergence rate of its maximum likelihood estimate is $n^{-1/4}$ rather than $n^{-1/2}$, while the mixing distribution has a convergence rate $n^{-1/8}$ rather than $n^{-1/4}$. This is in sharp contrast to ordinary semi-parametric models and to mixture models without a structural parameter.

Key words and phrases: Asymptotic distribution, finite mixture, Gaussian process, genetic analysis, likelihood ratio, non-regular model, semi-parametric model.