Computing the solution to a stochastic optimal control problem is difficult. A method of approximating a solution to a given stochastic optimal control problem using Markov chains was developed in [1]. This paper describes a suite of MATLAB functions implementing this method of approximating a...

Computing a numerical solution to a periodic optimal control problem is difficult. A method of approximating a solution to a given (stochastic) optimal control problem using Markov chains was developed in [3]. This paper describes an attempt at applying this method to a periodic optimal control...

This paper describes a suite of MATLAB routines devised to provide an approximately optimal solution to an infinite-horizon stochastic optimal control problem. The suite is an updated version of that described in [Kra01b]. Its routines implement a policy improvement algorithm to optimise a...

In this report, we outline a method for approximating a Markovian (or feedback-Nash) equilibrium of a dynamic game, possibly subject to coupled-constraints. We treat such a game as a "multiple" optimal control problem. A method for approximating a solution to a given optimal control problem via...

This paper is a successor of [AK08]. Both papers describe the same suite of MATLAB R° routines devised to provide an approximately optimal solution to an infinite horizon stochastic optimal control problem. The difference is that this paper explains how to allow for state and control...

This article is a modified version of [AK06]. Both articles explain how a suite of MATLAB routines distributed under the generic name SOCSol can be used to obtain optimal solutions to continuous-time stochastic optimal control problems. The difference between the SOCSol suites described by the...

Parallel MATLAB is a recent MathWorks product enabling the use of parallel computing methods on multicore personal computers. SOCSol is the generic name of a suite of MATLAB routines that can be used to obtain optimal solutions to continuous-time stochastic optimal control problems. In this...

Strategies for constructing a Markov decision chain approximating a continuous-time finite-horizon optimal control problem are investigated. Some simple, analytically soluble, examples are treated and low computational complexity is reported. Extensions to the method and implementation are...