Shrinkage of Weights Towards Subset Selection in Forecast Combination

We propose Hybrid Shrink (HS), a novel model for combining forecasts that merges forecast selection, weighting, and shrinkage concepts by shrinking forecasters’ weights to average weights, i.e., toward subset selection, or to zero. HS learns the forecast shrinkage direction by minimizing the Mean Squared Error (MSE) of the combined forecast by past error data plus two penalty terms. Two hyperparameters control the ratio of MSE to penalty cost and the number of forecasts being shrunk to their subset average. We further propose a novel procedure for hyperparameter tuning that learns shrinkage paths using all available training data, while shrinkage levels are subsequently determined by cross-validation. HS performance is evaluated using a comprehensive set of simulated forecast error data. The results show that HS can reduce MSE compared to approaches such as simple averaging, performing subset averaging, or using shrinkage towards average weights and standard cross-validation to learn and shrink weights