Adaptive extensions of the Nelder and Mead Simplex Method for optimization of stochastic simulation models

We consider the Nelder and Mead Simplex Method for the optimization of stochastic simulation models. Existing and new adaptive extensions of the Nelder and Mead simplex method designed to improve the accuracy and consistency of the observed best point are studied. We compare the performance of the extensions on a small microsimulation model, as well as on five test functions. We found that gradually decreasing the noise during an optimization run is the most preferred approach for stochastic objective functions. The amount of computation effort needed for successful optimization is very sensitive to the timing of noise reduction and to the rate of decrease of the noise. Restarting the algorithm during the optimization run, in the sense that the algorithm applies a fresh simplex at certain iterations during an optimization run, has adverse effects in our tests for the microsimulation model and for most test functions.