Numerical methods for the optimization of nonlinear stochastic delay systems, and an application to internet regulation

The Markov chain approximation method is an effective and widely used approach for computing optimal values and controls for stochastic systems. It was extended to nonlinear (and possibly reflected) diffusions with delays in a recent book. The path, control, and reflection terms can all be delayed. In models of communications systems, reflection terms correspond to buffer overflows, and to the adjustments needed to keep the buffers and rates nonnegative and bounded. The book concentrated on convergence proofs for general algorithms of promising forms. If the control and/or reflection terms are delayed, the memory requirements can make the problem intractable, since these have little or no regularity. For such models, the problem was recast in terms of a "wave equation, " to get practical algorithms with very much reduced computational needs. The overall aproach to getting usable algorithms is outlined. The methods are applied to an ideal model of a communications system with ...

• Type: Journal
• Year: 2010
• Where: CDC
• Authors: Harold J. Kushner
• Keywords: CDC 2010, Control Systems, Optimal Controls, Reflection Terms, Stochastic Systems