P L ] 1 0 N ov 2 01 6 Evolving the Incremental λ Calculus into a Model of Forward AD ∗

Formal transformations somehow resembling the usual derivative are surprisingly common in computer science, with two notable examples being derivatives of regular expressions [1] and derivatives of types [2, 3]. A newcomer to this list is the incremental λ-calculus, or ILC, a “theory of changes” that deploys a formal apparatus allowing the automatic generation of efficient update functions which perform incremental computation [4]. An example of this would be using the ILC derivative-like operator D to alter a function f : B → B, which performs some major reorganization on a database (of type B), into the update function D f : B → ∆B → ∆B. Here ∆B is the type of changes to B. So D f , given an initial database, maps a change to that input database to a change to the output database. This in principle, and as shown in their work also in practice, allows enormous savings when the change to the input is small compared to the size of the input itself. Resemblance to the standard derivative can be exhibited by a simple example