Sat 15 May 2010
Reverse-Mode Automatic Differentiation in Haskell
Posted by Edward Kmett under Algorithms , Haskell , Mathematics1 Comment
I've uploaded a package named rad to Hackage for handling reverse-mode automatic differentiation in Haskell.
Internally, it leverages a trick from Andy Gill's Kansas Lava to observe sharing in the tape it records for back propagation purposes, and uses type level branding to avoid confusing sensitivities.
I've tried to keep the API relatively close to that of Barak Pearlmutter and Jeffrey Mark Siskind's fad package, but I couldn't resist making a couple of minor tweaks here and there for generality.
I still need to go through and finish up the remaining unimplemented fad combinators, figure out a nice way to build a reverse-mode AD tower, validate that I didn't screw up my recollection of basic calculus, and provide a nice API for using this approach to get local reverse mode checkpoints in an otherwise forward mode AD program, but I am quite happy with how things have progressed thus far.
[Edit: I've uploaded minor bug fixes for exp and (**)]
May 22nd, 2010 at 5:01 pm
This is great stuff.
For the interested reader, see http://github.com/qobi/R6RS-AD for something similar, albeit a bit more shaken down, in Scheme.