Confidence sets based on penalized maximum likelihood estimators

The finite-sample coverage properties of confidence intervals based on penalized maximum likelihood estimators like the LASSO, adaptive LASSO, and hard-thresholding are analyzed. It is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the maximum likelihood estimator. In the case where the penalized estimators are tuned to possess the `sparsity property', the intervals based on these estimators are larger than the standard interval by an order of magnitude. A simple asymptotic confidence interval construction in the `sparse' case, that also applies to the smoothly clipped absolute deviation estimator, is also discussed.