The role of the Hessian matrix in fitting models to measurements

A numerical model can be fit to data by minimizing a positive quadratic function of the differences between the data and their model counterparts. The rate at which algorithms for computing the best fit to data converge depends on the size of the condition number and the distribution of eigenvalues of the Hessian matrix, which contains the second derivatives of this quadratic function. The inverse of the Hessian can be identified as the covariance matrix that establishes the accuracy to which the model state is determined by the data; the reciprocals of the Hessian's eigenvalues represent the variances of linear combinations of variables determined by its eigenvectors. The aspect of the model state that are most difficult to compute are those about which the data provide the least information. A unified formalism is presented in which the model may be treated as providing either strong or weak constraints, and methods for computing and inverting the Hessian matrix are discussed. Examples are given of the uncertainties resulting from fitting an oceanographic model to several different sets of hypothetical data.