Hidden Markov Model for Financial Time Series and Its Application to S&P 500 Index

The R package ldhmm is developed for the study of financial time series using Hidden Markov Model (HMM) with the lambda distribution framework. In particular, S&P 500 index is studied in depth due to its importance in finance and its long history. Major features in the index, such as regime identification, volatility clustering, and anti-correlation between return and volatility, can be extracted from HMM. Univariate symmetric lambda distribution is essentially a location-scale family of power-exponential distribution. Such distribution is suitable for describing highly leptokurtic time series in the financial market. It provides a theoretically solid foundation to explore such data where the normal distribution may not be adequate. The index is analyzed from two states to six states, then ten states. The five-state HMM and above can capture large amount of auto-correlation, matching what's observed in the data. This is a major validation for the HMM. Although the stock market can be broadly classified into the normal regime and the crash regime, The progression of HMM states allows to go beyond the two-regime paradigm. The index history can be decomposed to a spectrum of volatility states. And the trend of the mean and volatility in HMM states confirms the recognized fact that the stock market tends to rise when the volatility is low, while tends to fall when the volatility is high. The pivotal volatility is calculated. Specifically, we compare the expected volatility from the ten-state HMM to both the VIX index with an adjustment factor and the realized volatility from Oxford-Man Realized Library. They match quite well. This indicates high-state HMM can serve as a tool for volatility forecasting