Many important models utilize estimation of average derivatives of the conditional mean function. Asymptotic results in the literature on density weighted average derivative estimators (ADE) focus on convergence at parametric rates; this requires making stringent assumptions on smoothness of the underlying density; here we derive asymptotic properties under relaxed smoothness assumptions. We adapt to the unknown smoothness in the model by consistently estimating the optimal bandwidth rate and using linear combinations of ADE estimators for different kernels and bandwidths. Linear combinations of estimators (i) can have smaller asymptotic mean squared error (AMSE) than an estimator with an optimal bandwidth and (ii) when based on estimated optimal rate bandwidth can adapt to unknown smoothness and achieve rate optimality. Our combined estimator minimizes the trace of estimated MSE of linear combinations. Monte Carlo results for ADE confirm good performance of the combined estimator.
