Periodically homogeneous Markov chains: The discrete state space case

A unified theory of periodically homogeneous Markov chains on countable state spaces with periodically time-varying transition probabilities is introduced. The finite-dimensional probability distributions of these time-periodic chains are first studied and their correspondence with the marginal distributions and transition probabilities is shown. Then, the concepts of periodic stability/regularity and limiting behaviors are proposed. The communicability and classification of states necessary for establishing periodic stability are then examined. In particular, periodic irreducibility and the main solidarity/class properties are presented, namely periodic recurrence, periodic positive recurrence, periodic transience, and periodic aperiodicity. Furthermore, sufficient conditions for periodic stochastic stability of time-periodic Markov chains are derived. Finally, various applications to some operations research models and time series analysis are considered. In particular, periodic Markov decision processes, periodic integer-valued time series models, and periodic Markov-switching time series models are examined.