This paper extends the class of deterministic volatility Heath-Jarrow-Morton (1992) models to a Markov chain stochastic volatility framework allowing for jump discontinuities and a variety of deformations of the term structure of forward rate volatilities. Analytical solutions for the dynamics of the volatility term structure are obtained. Semimartingale decompositions of the interest rates under a spot and forward martingale measures are identified. Stochastic volatility versions of the continuous time Ho-Lee (1986) and Hull-White (1990, 1993) extended Vasicek(1977) models are obtained. Introducing a regime shift in volatility that is an exponential function of time to maturity leads to a Vasicek dynamics with regime switching coeffcients of the short rate.