Optimization of the Transient and Steady-State Behavior of Discrete Event Systems

We present a general framework for applying simulation to optimize the behavior of discrete event systems. Our approach involves modeling the discrete event system under study as a general state space Markov chain whose distribution depends on the decision parameters. We then show how simulation and the likelihood ratio method can be used to evaluate the performance measure of interest and its gradient, and we present conditions that guarantee that the Robbins-Monro stochastic approximation algorithm will converge almost surely to the optimal values of the decision parameters. Both transient and steady-state performance measures are considered. For steady-state performance measures, we consider both the case when the Markov chain of interest is regenerative in the standard sense, as well as the case when this Markov chain is Harris recurrent, and thereby regenerative in a wider sense.