We develop a methodology for studying “large deviations type” questions. Our approach does not require that the large deviations principle holds, and is thus applicable to a large class of systems. We study a system of queues with exponential servers, which share an arrival stream. Arrivals...

We develop a methodology for studying “large deviations type” questions. Our approach does not require that the large deviations principle holds, and is thus applicable to a large class of systems. We study a system of queues with exponential servers, which share an arrival stream. Arrivals...

Recursive parameter estimation in diffusion processes is considered. First, stability and asymptotic properties of the global, off-line MLE (maximum likelihood estimator) are obtained under explicit conditions. The MLE evolution equation is then derived by employing a generalized Itô...

Results on the convergence with probability one of stochastic approximation algorithms of the form [theta]n+1 = [theta]n - [gamma]n+1 h([theta]n) + un+1 are given, where the [theta]'s belong to some Banach space and {un} is a stochastic process. Using this extension of results of Kushner and...

The theory of Markov Decision Processes - also known under several other names including sequential stochastic optimization, discrete-time stochastic control, and stochastic dynamic programming - studies sequential optimization of discrete time stochastic systems. Fundamentally, this is a...

This paper applies importance sampling simulation for estimating rare event probabilities of the first passage time in the infinite server queue with renewal arrivals and general service time distributions. We consider importance sampling algorithms which are based on large deviations results of...

There are various importance sampling schemes to estimate rare event probabilities in Markovian systems such as Markovian reliability models and Jackson networks. In this work, we present a general state-dependent importance sampling method which partitions the state space and applies the...