9. Secant Methods for Unconstrained Minimization

In this chapter we consider secant methods for the unconstrained minimization problem. The derivatives we have used in our algorithms for this problem are the gradient, ∇ƒ(x), and the Hessian, . The gradient must be known accurately in minimization algorithms, both for calculating descent directions and for stopping tests, and the reader can see from Chapter 8 that secant approximations do not provide this accuracy. Therefore, secant approximations to the gradient are not used in quasi-Newton algorithms. On the other hand, the Hessian can be approximated by secant techniques in much the same manner as the Jacobian was in Chapter 8, and this is the topic of the present chapter. We will present the most successful secant updates to the Hessian and the theory that accompanies them. These updates require no additional function or gradient evaluations, and again lead to locally q-superlinearly convergent algorithms.