Maximum entropy derivation of quasi-Newton methods

In this work we re-derive and improve upon quasi-Newton methods commonly used to find the zeros or extrema of functions, using the maximum entropy method.Unlike Newton's method, in which the Jacobian or Hessian matrix is calculated at each iteration, quasi-Newton methods find an approximation to the matrix by updating it from the previous iteration.The updates generally follow the under-determined secant equation.The methodology used here differs from previous maximum entropy quasi-Newton derivations found in the literature, in that it updates the average values of the Jacobian or Hessian rather than updating a covariance matrix while keeping the mean fixed at zero.By approaching the derivation differently to previous studies, new insights were obtained into how quasi-Newton methods behave and how they can be improved.Numerical experiments demonstrate several improvements on existing quasi-Newton methods.