The ad package

[Tags: bsd3, library]

Forward-, reverse- and mixed- mode automatic differentiation combinators with a common API.

Type-level "branding" is used to both prevent the end user from confusing infinitesimals and to limit unsafe access to the implementation details of each Mode.

Each mode has a separate module full of combinators.

While not every mode can provide all operations, the following basic operations are supported, modified as appropriate by the suffixes below:

The following suffixes alter the meanings of the functions above as follows:


[Skip to ReadMe]

Properties

Versions0.12, 0.13, 0.15, 0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23, 0.24, 0.27, 0.28, 0.30.0, 0.31.0, 0.32.0, 0.33.0, 0.40, 0.40.1, 0.44.0, 0.44.1, 0.44.2, 0.44.3, 0.44.4, 0.45.0, 0.46.0, 0.46.1, 0.46.2, 0.47.0, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6, 1.1.0, 1.1.0.1, 1.1.1, 1.1.3, 1.2.0, 1.2.0.1, 1.2.0.2, 1.3, 1.3.0.1, 1.3.1, 1.4, 1.5, 1.5.0.1, 1.5.0.2, 3.0, 3.0.1, 3.1.1, 3.1.2, 3.1.3, 3.1.4, 3.2, 3.2.1, 3.2.2, 3.3.0.1, 3.3.1, 3.3.1.1, 3.4, 4.0, 4.0.0.1, 4.1, 4.2, 4.2.0.1, 4.2.1, 4.2.1.1, 4.2.2, 4.2.3
Change logCHANGELOG.markdown
Dependenciesarray (>=0.2 && <0.6), base (>=4.3 && <5), comonad (==4.*), containers (>=0.2 && <0.6), data-reify (==0.6.*), erf (==2.0.*), free (>=4.6.1 && <5), nats (>=0.1.2 && <2), reflection (>=1.4 && <3), tagged (>=0.7 && <1), transformers (>=0.3 && <0.5) [details]
LicenseBSD3
Copyright(c) Edward Kmett 2010-2015, (c) Barak Pearlmutter and Jeffrey Mark Siskind 2008-2009
AuthorEdward Kmett
Maintainerekmett@gmail.com
StabilityExperimental
CategoryMath
Home pagehttp://github.com/ekmett/ad
Bug trackerhttp://github.com/ekmett/ad/issues
Source repositoryhead: git clone git://github.com/ekmett/ad.git
UploadedFri Jul 10 04:34:40 UTC 2015 by EdwardKmett
DistributionsLTSHaskell:4.2.3, NixOS:4.2.3, Stackage:4.2.3
Downloads12789 total (643 in last 30 days)
Votes
1 []
StatusDocs available [build log]
Last success reported on 2015-07-10 [all 1 reports]

Modules

[Index]

Downloads

Maintainers' corner

For package maintainers and hackage trustees

Readme for ad-4.2.3

ad

Build Status

A package that provides an intuitive API for Automatic Differentiation (AD) in Haskell. Automatic differentiation provides a means to calculate the derivatives of a function while evaluating it. Unlike numerical methods based on running the program with multiple inputs or symbolic approaches, automatic differentiation typically only decreases performance by a small multiplier.

AD employs the fact that any program y = F(x) that computes one or more value does so by composing multiple primitive operations. If the (partial) derivatives of each of those operations is known, then they can be composed to derive the answer for the derivative of the entire program at a point.

This library contains at its core a single implementation that describes how to compute the partial derivatives of a wide array of primitive operations. It then exposes an API that enables a user to safely combine them using standard higher-order functions, just as you would with any other Haskell numerical type.

There are several ways to compose these individual Jacobian matrices. We hide the choice used by the API behind an explicit "Mode" type-class and universal quantification. This prevents users from confusing infinitesimals. If you want to risk infinitesimal confusion in order to get greater flexibility in how you curry, flip and generally combine the differential operators, then the Rank1.* modules are probably your cup of tea.

Features

Examples

You can compute derivatives of functions

Prelude Numeric.AD> diff sin 0 {- cos 0 -}
1.0

Or both the answer and the derivative of a function:

Prelude Numeric.AD> diff' (exp . log) 2
(2.0,1.0)

You can compute the derivative of a function with a constant parameter using auto from Numeric.AD.Types:

Prelude Numeric.AD Numeric.AD.Types> let t = 2.0 :: Double
Prelude Numeric.AD Numeric.AD.Types> diff (\ x -> auto t * sin x) 0
2.0

You can use a symbolic numeric type, like the one from simple-reflect to obtain symbolic derivatives:

Prelude Debug.SimpleReflect Numeric.AD> diff atanh x
recip (1 - x * x) * 1

You can compute gradients for functions that take non-scalar values in the form of a Traversable functor full of AD variables.

Prelude Numeric.AD Debug.SimpleReflect> grad (\[x,y,z] -> x * sin (x + log y)) [x,y,z]
[ 0 + (0 + sin (x + log y) * 1 + 1 * (0 + cos (x + log y) * (0 + x * 1)))
, 0 + (0 + recip y * (0 + 1 * (0 + cos (x + log y) * (0 + x * 1))))
, 0
]

which one can simplify to:

[ sin (x + log y) + cos (x + log y) * x, recip y * cos (x + log y) * x, 0 ]

If you need multiple derivatives you can calculate them with diffs:

Prelude Numeric.AD> take 10 $ diffs sin 1
[0.8414709848078965,0.5403023058681398,-0.8414709848078965,-0.5403023058681398,0.8414709848078965,0.5403023058681398,-0.8414709848078965,-0.5403023058681398,0.8414709848078965,0.5403023058681398]

or if your function takes multiple inputs, you can use grads, which returns an 'f-branching stream' of derivatives, that you can inspect lazily. Somewhat more intuitive answers can be obtained by converting the stream into the polymorphically recursive Jet data type. With that we can look at a single "layer" of the answer at a time:

The answer:

Prelude Numeric.AD Numeric.AD.Types> headJet $ jet $  grads (\[x,y] -> exp (x * y)) [1,2]
7.38905609893065

The gradient:

Prelude Numeric.AD Numeric.AD.Types> headJet $ tailJet $ jet $  grads (\[x,y] -> exp (x * y)) [1,2]
[14.7781121978613,7.38905609893065]

The hessian (n * n matrix of 2nd derivatives)

Prelude Numeric.AD Numeric.AD.Types> headJet $ tailJet $ tailJet $ jet $  grads (\[x,y] -> exp (x * y)) [1,2]
[[29.5562243957226,22.16716829679195],[22.16716829679195,7.38905609893065]]

Or even higher order tensors of derivatives as a jet.

Prelude Numeric.AD Numeric.AD.Types> headJet $ tailJet $ tailJet $ tailJet $ jet $  grads (\[x,y] -> exp (x * y)) [1,2]
[[[59.1124487914452,44.3343365935839],[44.3343365935839,14.7781121978613]],[[44.3343365935839,14.7781121978613],[14.7781121978613,7.38905609893065]]]

Note the redundant values caused by the various symmetries in the tensors. The ad library is careful to compute each distinct derivative only once, lazily and to share the resulting computation.

Overview

Modules

Combinators

While not every mode can provide all operations, the following basic operations are supported, modified as appropriate by the suffixes below:

Combinator Suffixes

The following suffixes alter the meanings of the functions above as follows:

Contact Information

Contributions and bug reports are welcome!

Please feel free to contact me through github or on the #haskell IRC channel on irc.freenode.net.

-Edward Kmett