Solutions and diagnostics of switching problems with linear state dynamics

Juri Hinz, Nicholas Yap


Optimal control problems of stochastic switching type appear frequently when making decisions under uncertainty and are notoriously challenging from a computational viewpoint. Although numerous approaches have been suggested in the literature to tackle them, typical real-world applications are inherently high dimensional and usually drive common algorithms to their computational limits. Furthermore, even when numerical approximations of the optimal strategy are obtained, practitioners must apply time-consuming and unreliable Monte Carlo simulations to assess their quality. In this paper, we show how one can overcome both difficulties for a specific class of discrete-time stochastic control problems. A simple and efficient algorithm which yields approximate numerical solutions is presented and methods to perform diagnostics are provided.



Markov decisions; approximate dynamic programming; stochastic control; duality


Remember, for most actions you have to record/upload into this online system
and then inform the editor/author via clicking on an email icon or Completion button.
ANZIAM Journal, ISSN 1446-8735, copyright Australian Mathematical Society.