Design Adaptive Nearest Neighbor Regression Estimation

This paper deals with nonparametric regression estimation under arbitrary sampling with an unknown distribution. The effect of the distribution of the design, which is a nuisance parameter, can be eliminated by conditioning. An upper bound for the conditional mean squared error of k-NN estimates leads us to consider an optimal number of neighbors, which is a random function of the sampling. The corresponding estimate can be used for nonasymptotic inference and is also consistent under a minimal recurrence condition. Some deterministic equivalents are found for the random rate of convergence of this optimal estimate, for deterministic and random designs with vanishing or diverging densities. The proposed estimate is rate optimal for standard designs.