On a Class of Stochastic Optimization Problems with a Specified Growth Pattern

We consider a system which consists of several subsystems. The outputs of these subsystems satisfy linear difference equations which specify the growth pattern of the output of the system over time. The state of each subsystem is described by a finite Markov chain, the transition probabilities of which are subject to our control. Associated with the Markov chain of each subsystem is a cost per unit output of the subsystem, and the cost is incurred as the subsystem occupies one of J states in each epoch. The problem of minimizing the total expected cost with respect to the transition probabilities over a sufficiently long period of time is shown under certain conditions to reduce to a collection of n independent programs. Each of these can be solved by column generation techniques.