The Art of Differentiating Computer Programs

This is the first entry-level book on algorithmic (also known as automatic) differentiation (AD), providing fundamental rules for the generation of first- and higher-order tangent-linear and adjoint code. The author covers the mathematical underpinnings as well as how to apply these observations to real-world numerical simulation programs. Readers will find many examples and exercises, including hints to solutions. Also included are the prototype AD tools dco and dcc for use with the examples and exercises. The derivative code compiler dcc provides first- and higher-order tangent-linear and adjoint modes for a limited subset of C/C++. In addition, readers will have access to a supplementary website containing sources of all software discussed in the book, additional exercises and comments on their solutions (growing over the coming years), links to other sites on AD, and errata. Audience: This book is intended for undergraduate and graduate students in computational science, engineering, and finance as well as applied mathematics and computer science. It will provide researchers and developers at all levels with an intuitive introduction to AD.