NONPARAMETRIC TECHNIQUES IN FINITE MIXTURE OF REGRESSION MODELS

Mixture models have been popular in the literature of both statistics and social science. In this dissertation, we propose a new mixture model, namely, nonparametric finite mixture of regression models, which can be viewed as a natural extension of finite mixture of linear regression. In the newly proposed model, it allows both the regression and variance function as functions of covariates, and their functional forms are nonparametric rather than a specified form. We first consider the mixing proportion in the nonparametric finite mixture of regression models is also a nonparametric function of covariates. We develop an estimation procedure for the nonparametric finite mixture of regression models by employing kernel regression, and proposed an algorithm to carry out the estimation procedure by modifying an EM algorithm. We further systematically studied the sampling properties of the newly proposed estimation procedures and the proposed algorithm. We found that the proposed algorithm preserves the ascent property of the EM algorithm in an asymptotic sense. We derive the asymptotic bias and variance of the resulting estimate. We further established the asymptotic normality of the resulting estimate. Monte Carlo simulation studies are conducted to assess the finite sample performance of the resulting estimate. The proposed methodology is illustrated by analysis of a real data example.We further study the nonparametric finite mixture of regression models with constant mixing proportion. Since the mixing proportion is parametric, while the regression function and variance function for each components are nonparametric, the model indeed is a semiparametric model. To achieve better convergent rate for mixing proportional parameters, we develop an estimation procedures by using back-fitting algorithm. To reduce computational cost, we further suggest one-step back-fitting algorithm, which behaves similar to the gradient ECM algorithm. Thus, the convergence behavior of the proposed algorithm can be analyzed along the lines for the gradient EM algorithm. We studied the asymptotic propertiesof the resulting estimate. We showed that the resulting estimate for the mixing proportion parameter is root $n$ consistent, and follows an asymptotic normal distribution. We also derived the asymptotic bias and variance for the resulting estimate of the regression function and variance function, and further established their asymptotic normality. Finite sample performance of the proposed procedure is examined by a Monte Carlo simulation study. The proposed procedure is demonstrated by analysis of a real data example.As the advent of data collection technology and data storage device, researchers are able to collect functional data without much cost. In this dissertation, we studied mixture models for functional data. More specifically, we proposed mixtures of Gaussian processes for functional data. The proposed model is a natural extension of mixture of high-dimensional normals. We develop an estimation procedure to the mean and covariance function of mixture of Gaussian processes by using kernel regression. The proposed methodology is empirically justified by simulation and illustrated by an analysis of the supermarket data.