Automatic Differentiation of Algorithms: From Simulation to Optimization

A survey book focusing on the key relationships and synergies between automatic differentiation (AD) tools and other software tools, such as compilers and parallelizers, as well as their applications. The key objective is to survey the field and present the recent developments. In doing so the topics covered shed light on a variety of perspectives. They reflect the mathematical aspects, such as the differentiation of iterative processes, and the analysis of nonsmooth code. They cover the scientific programming aspects, such as the use of adjoints in optimization and the propagation of rounding errors. They also cover "implementation" problems.