Stochastic Optimal Control Problems with a Bounded Memory

Abstract This paper treats a finite time horizon optimal control problem in which the controlled state dynamics is governed by a general system of stochastic functional differential equations with a bounded memory. An infinite-dimensional HJB equation is derived using a Bellman-type dynamic programming principle. It is shown that the value function is the unique viscosity solution of the HJB equation. In addition, the computation issues are also studied. More particularly, a finite difference scheme is obtained to approximate the viscosity solution of the infinite dimensional HJB equation. The convergence of the scheme is proved using the Banach fixed point theorem. The computational algorithm is also provided based on the scheme obtained.