An EM algorithm for continuous-time bivariate Markov chains

We study properties and parameter estimation of a finite-state, homogeneous, continuous-time, bivariate Markov chain. Only one of the two processes of the bivariate Markov chain is assumed observable. The general form of the bivariate Markov chain studied here makes no assumptions on the structure of the generator of the chain. Consequently, simultaneous jumps of the observable and underlying processes are possible, neither process is necessarily Markov, and the time between jumps of each of the two processes has a phase-type distribution. Examples of bivariate Markov chains include the Markov modulated Poisson process and the batch Markovian arrival process when appropriate modulo counts are used in each case. We develop an expectation–maximization (EM) procedure for estimating the generator of a bivariate Markov chain, and we demonstrate its performance. The procedure does not rely on any numerical integration or sampling scheme of the continuous-time bivariate Markov chain. The proposed EM algorithm is equally applicable to multivariate Markov chains.