On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification

Let be identically distributed random vectors in , independently drawn according to some probability density. An observation is said to be a layered nearest neighbour (LNN) of a point if the hyperrectangle defined by and contains no other data points. We first establish consistency results on , the number of LNN of . Then, given a sample of independent identically distributed random vectors from , one may estimate the regression function by the LNN estimate , defined as an average over the Yi's corresponding to those which are LNN of . Under mild conditions on r, we establish the consistency of towards 0 as n-->[infinity], for almost all and all p>=1, and discuss the links between rn and the random forest estimates of Breiman (2001) [8]. We finally show the universal consistency of the bagged (bootstrap-aggregated) nearest neighbour method for regression and classification.