This book presents recent findings and results concerning the solutions of especially finite state-space Markov decision problems and determining Nash equilibria for related stochastic games with average and total expected discounted reward payoffs. In addition, it focuses on a new class of...

We develop a quantitative framework for exploring how individuals trade off the utility benefit of social activity against the internal and external health risks that come with social interactions during a pandemic. We calibrate the model to external targets and then compare its predictions with...

The social cost of carbon is the expected present value of damages from emitting one ton of carbon today. We use perturbation theory to derive an approximate tractable expression for this cost adjusted for climatic and economic risk. We allow for different aversion to risk and intertemporal...

The optimal policy response to a low-probability extreme event is examined. A simple policy problem is solved for a sequence of different loss functions: quadratic, combined quadratic/absolute-deviation, absolute-deviation, combined quadratic/constant, and perfectionist. The paper shows that,...

This paper presents a framework for understanding the limits that exist in optimal policy design in dynamic contexts. We consider the design of policies in the context of dynamic linear models. Fundamental design limits exist for policy rules in such environments in the sense that any policy...

In this paper we explore some of the benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems. We first show that the implicit finite-difference scheme of Achdou et al. (2017) amounts to a limiting...

It is frequently observed that the implementation of green policies is delayed compared to the initial announcement. Considering a setting with a representative monopolist extracting a non-renewable resource, we demonstrate that announcing a green policy, but then delaying its implementation, is...

We analyze a W-configuration assemble-to-order system with random lead times, random arrival of demand, and lost sales, in continuous time. Specifically, we assume exponentially distributed production and demand inter-arrival times. We formulate the problem as an infinite-horizon Markov decision...