Bayesian Estimation and Optimization for Learning Sequential Regularized Portfolios

This paper incorporates Bayesian estimation and optimization into portfolio selection framework, particularly for high-dimensional portfolio in which the number of assets is larger than the number of observations. We leverage a constrained 𝓁1 minimization approach, called linear programming optimal (LPO) portfolio, to directly estimate effective parameters appearing in the optimal portfolio. We propose two refinements for the LPO strategy. First, we explore improved Bayesian estimates, instead of sample estimates, of the covariance matrix of asset returns. Second, we introduce Bayesian optimization (BO) to replace traditional grid-search cross-validation (CV) in tuning hyperparameters of the LPO strategy. We further propose modifications in the BO algorithm by (1) taking into account time-dependent nature of financial problems and (2) extending commonly used expected improvement (EI) acquisition function to include a tunable trade-off with the improvement's variance (EIVar). Allowing a general case of noisy observations, we theoretically derive the sub-linear convergence rate of BO under the newly proposed EIVar and thus our algorithm has no regret. Our empirical studies confirm that the adjusted BO result in portfolios with higher out-of-sample Sharpe ratio, certainty equivalent, and lower turnover compared to those tuned with CV. This superior performance is achieved with significant reduction in time elapsed, thus also addressing time-consuming issues of CV. Furthermore, LPO with Bayesian estimates outperform original proposal of LPO, as well as the benchmark equally weighted and plug-in strategies