We consider a finite-state, finite-action, infinite-horizon, discounted reward Markov decision process and study the bias and variance in the value function estimates that result from empirical estimates of the model parameters. We provide closed-form approximations for the bias and variance,...

We study a single product inventory system with nonnegative setup cost in which the demand is a continuous random variable but orders are restricted to be integer valued. Optimal policies, when there are no setup costs, have a nice form. However, we show that, when the setup costs are nonzero,...

Deciding who should receive a mail-order catalog is among the most important decisions that mail-order-catalog firms must address. In practice, the current approach to the problem is invariably myopic: firms send catalogs to customers who they think are most likely to order from that catalog. In...