Statistical Analysis of Curved Probability Densities

Suppose that pn(· ; [theta]) is the joint probability density of n observations which are not necessarily i.i.d. In this paper we discuss the estimation of an unknown parameter u of a family of "curved probability densities" defined by M = {pn(· ; [theta](u)), dim u < dim [theta]} embedded in S = {pn(· ; [theta]), [theta] [set membership, variant] [Theta]}, and develop the higher order asymptotic theory. The third-order Edgeworth expansion for a class of estimators is derived. It is shown that the maximum likelihood estimator is still third-order asymptotically optimal in our general situation. However, the Edgeworth expansion contains two terms which vanish in the case of curved exponential family. Regarding this point we elucidate some results which did not appear in Amari's framework. Our results are applicable to time series analysis and multivariate analysis. We give a few examples (e.g., a family of curved ARMA models, a family of curved regression models).