Automatic Differentiation

Differentiable programming is a process of computing derivative over functions automatically. These functions can operate on floating point values, tensors, and user-defined data structures containing them.

Background on Differentiable Programming

Below are some review papers and a book on differentiable programming. The field is also called automatic differentiation or algorithmic differentiation.

A Review of Automatic Differentiation and its Efficient Implementation, (2019)

Automatic Differentiation in Machine Learning: A Survey, (2018)

Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd Ed., (2008)