Solve the optimal control problem using policy iteration, i.e. find the optimal strategy and value function for the case where the uncontrolled system is given by a subgenerator G0 (either due to discounting or due to absorbing boundaries)

PolicyIterationRegular(
  G0,
  G1,
  k,
  uopt,
  iter.max = 1000,
  tol = 1e-12,
  do.minimize = TRUE,
  do.return.QSD = FALSE
)

Arguments

G0: The sub-generator of the uncontrolled system
G1: A list of (sub-)generators for each control
k: The running cost
uopt: A list of functions returning optional controls as function of
iter.max: = 1000 Maximum number of iterations
tol: = 1e-12 Tolerance for convergence
do.minimize=TRUE
do.return.QSD=FALSE: Compute and return the quasi-stationary distribution

Value

A list containing V: The value function, as a vector with an element for each state u: The optimal controls, as a matrix with a row for each state and a column for each control

and, if do.return.QSD==TRUE, qsd.value: The decay rate of the quasi-stationary distribution (decay rate) qsd.vector: The quasi-stationary distribution

Examples

## Controlling a system to the boundary with minimum effort
xi <- seq(-2,2,length=101)
xc <- as.numeric(cell.centers(xi,c(0,1))$x)
dx <- diff(xi)

G0 <- fvade(function(x)-x,function(x)1,xi,'a')
#> Loading required package: Matrix
Gp <- fvade(function(x)1,function(x)0,xi,'a')
Gn <- fvade(function(x)-1,function(x)0,xi,'a')

uopt <- function(dV)pmax(0,-dV)
k <- function(u) 1 + 0.5*u[,1]^2 + 0.5*u[,2]^2
sol <- PolicyIterationRegular(G0,list(Gp,Gn),k,list(uopt,uopt),do.return.QSD=TRUE)

par(mfrow=c(1,3))
plot(xc,sol$V,xlab="x",ylab="Value function",type="l")
plot(xc,sol$u[,1]-sol$u[,2],type="l",xlab="x",ylab="Optimal control")
plot(xc,sol$qsd.vector/dx,type="l",xlab="x",ylab="QSD",main=sol$qsd.value)