Portability | GHC only |
---|---|

Stability | experimental |

Maintainer | ekmett@gmail.com |

- Gradients (Reverse Mode)
- Jacobians (Mixed Mode)
- Monadic Gradient/Jacobian (Reverse Mode)
- Functorial Gradient/Jacobian (Reverse Mode)
- Transposed Jacobians (Forward Mode)
- Hessian (Forward-On-Reverse)
- Hessian Tensors (Forward-On-Mixed)
- Hessian Vector Products (Forward-On-Reverse)
- Higher Order Gradients/Hessians (Sparse Forward)
- Derivatives (Forward Mode)
- Derivatives (Tower)
- Directional Derivatives (Forward Mode)
- Directional Derivatives (Tower)
- Taylor Series (Tower)
- Maclaurin Series (Tower)
- Monadic Combinators (Forward Mode)
- Exposed Types

Mixed-Mode Automatic Differentiation.

Each combinator exported from this module chooses an appropriate AD mode.

- grad :: (Traversable f, Num a) => FU f a -> f a -> f a
- grad' :: (Traversable f, Num a) => FU f a -> f a -> (a, f a)
- gradWith :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> f b
- gradWith' :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> (a, f b)
- jacobian :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (f a)
- jacobian' :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (a, f a)
- jacobianWith :: (Traversable f, Traversable g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)
- jacobianWith' :: (Traversable f, Traversable g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)
- gradM :: (Traversable f, Monad m, Num a) => FF f m a -> f a -> m (f a)
- gradM' :: (Traversable f, Monad m, Num a) => FF f m a -> f a -> m (a, f a)
- gradWithM :: (Traversable f, Monad m, Num a) => (a -> a -> b) -> FF f m a -> f a -> m (f b)
- gradWithM' :: (Traversable f, Monad m, Num a) => (a -> a -> b) -> FF f m a -> f a -> m (a, f b)
- gradF :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (f a)
- gradF' :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (a, f a)
- gradWithF :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)
- gradWithF' :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)
- jacobianT :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> f (g a)
- jacobianWithT :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> f (g b)
- hessian :: (Traversable f, Num a) => FU f a -> f a -> f (f a)
- hessianTensor :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (f (f a))
- hessianProduct :: (Traversable f, Num a) => FU f a -> f (a, a) -> f a
- hessianProduct' :: (Traversable f, Num a) => FU f a -> f (a, a) -> f (a, a)
- gradients :: (Traversable f, Num a) => FU f a -> f a -> Stream f a
- diff :: Num a => UU a -> a -> a
- diffF :: (Functor f, Num a) => UF f a -> a -> f a
- diff' :: Num a => UU a -> a -> (a, a)
- diffF' :: (Functor f, Num a) => UF f a -> a -> f (a, a)
- diffs :: Num a => UU a -> a -> [a]
- diffsF :: (Functor f, Num a) => UF f a -> a -> f [a]
- diffs0 :: Num a => UU a -> a -> [a]
- diffs0F :: (Functor f, Num a) => UF f a -> a -> f [a]
- du :: (Functor f, Num a) => FU f a -> f (a, a) -> a
- du' :: (Functor f, Num a) => FU f a -> f (a, a) -> (a, a)
- duF :: (Functor f, Functor g, Num a) => FF f g a -> f (a, a) -> g a
- duF' :: (Functor f, Functor g, Num a) => FF f g a -> f (a, a) -> g (a, a)
- dus :: (Functor f, Num a) => FU f a -> f [a] -> [a]
- dus0 :: (Functor f, Num a) => FU f a -> f [a] -> [a]
- dusF :: (Functor f, Functor g, Num a) => FF f g a -> f [a] -> g [a]
- dus0F :: (Functor f, Functor g, Num a) => FF f g a -> f [a] -> g [a]
- taylor :: Fractional a => UU a -> a -> a -> [a]
- taylor0 :: Fractional a => UU a -> a -> a -> [a]
- maclaurin :: Fractional a => UU a -> a -> [a]
- maclaurin0 :: Fractional a => UU a -> a -> [a]
- diffM :: (Monad m, Num a) => UF m a -> a -> m a
- diffM' :: (Monad m, Num a) => UF m a -> a -> m (a, a)
- module Numeric.AD.Types
- class Lifted t => Mode t where

# Gradients (Reverse Mode)

grad :: (Traversable f, Num a) => FU f a -> f a -> f aSource

grad' :: (Traversable f, Num a) => FU f a -> f a -> (a, f a)Source

gradWith :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> f bSource

function calculates the gradient of a non-scalar-to-scalar function `grad`

g f`f`

with reverse-mode AD in a single pass.
The gradient is combined element-wise with the argument using the function `g`

.

grad == gradWith (\_ dx -> dx) id == gradWith const

gradWith' :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> (a, f b)Source

# Jacobians (Mixed Mode)

jacobian :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (f a)Source

Calculate the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.

If you need to support functions where the output is only a `Functor`

or `Monad`

, consider `Numeric.AD.Reverse.jacobian`

or `Numeric.AD.Reverse.gradM`

from Numeric.AD.Reverse.

jacobian' :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (a, f a)Source

Calculate both the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward- and reverse- mode AD based on the relative, number of inputs and outputs.

If you need to support functions where the output is only a `Functor`

or `Monad`

, consider `Numeric.AD.Reverse.jacobian'`

or `Numeric.AD.Reverse.gradM'`

from Numeric.AD.Reverse.

jacobianWith :: (Traversable f, Traversable g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)Source

calculates the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.
`jacobianWith`

g f

The resulting Jacobian matrix is then recombined element-wise with the input using `g`

.

If you need to support functions where the output is only a `Functor`

or `Monad`

, consider `Numeric.AD.Reverse.jacobianWith`

or `Numeric.AD.Reverse.gradWithM`

from Numeric.AD.Reverse.

jacobianWith' :: (Traversable f, Traversable g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)Source

calculates the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.
`jacobianWith'`

g f

The resulting Jacobian matrix is then recombined element-wise with the input using `g`

.

If you need to support functions where the output is only a `Functor`

or `Monad`

, consider `Numeric.AD.Reverse.jacobianWith'`

or `Numeric.AD.Reverse.gradWithM'`

from Numeric.AD.Reverse.

# Monadic Gradient/Jacobian (Reverse Mode)

gradWithM' :: (Traversable f, Monad m, Num a) => (a -> a -> b) -> FF f m a -> f a -> m (a, f b)Source

# Functorial Gradient/Jacobian (Reverse Mode)

gradF :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (f a)Source

The `gradF`

function calculates the jacobian of a non-scalar-to-non-scalar function with reverse AD lazily in `m`

passes for `m`

outputs.

gradWithF :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)Source

'gradWithF g f' calculates the Jacobian of a non-scalar-to-non-scalar function `f`

with reverse AD lazily in `m`

passes for `m`

outputs.

Instead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the `g`

.

gradF == gradWithF (\_ dx -> dx) gradWithF const == (\f x -> const x <$> f x)

gradWithF' :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)Source

`gradWithF`

g f' calculates both the result and the Jacobian of a nonscalar-to-nonscalar function `f`

, using `m`

invocations of reverse AD,
where `m`

is the output dimensionality. Applying `fmap snd`

to the result will recover the result of `gradWithF`

Instead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the `g`

.

jacobian' == gradWithF' (\_ dx -> dx)

# Transposed Jacobians (Forward Mode)

jacobianT :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> f (g a)Source

A fast, simple transposed Jacobian computed with forward-mode AD.

jacobianWithT :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> f (g b)Source

A fast, simple transposed Jacobian computed with forward-mode AD.

# Hessian (Forward-On-Reverse)

hessian :: (Traversable f, Num a) => FU f a -> f a -> f (f a)Source

Compute the hessian via the jacobian of the gradient. gradient is computed in reverse mode and then the jacobian is computed in forward mode.

# Hessian Tensors (Forward-On-Mixed)

hessianTensor :: (Traversable f, Traversable g, Num a) => FF f g a -> f a -> g (f (f a))Source

Compute the order 3 Hessian tensor on a non-scalar-to-non-scalar function via the forward-mode Jacobian of the mixed-mode Jacobian of the function.

# Hessian Vector Products (Forward-On-Reverse)

hessianProduct :: (Traversable f, Num a) => FU f a -> f (a, a) -> f aSource

computes the product of the hessian `hessianProduct`

f wv`H`

of a non-scalar-to-scalar function `f`

at `w = `

with a vector `fst`

$ wv`v = snd $ wv`

using "Pearlmutter's method" from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.6143, which states:

H v = (d/dr) grad_w (w + r v) | r = 0

Or in other words, we take the directional derivative of the gradient.

hessianProduct' :: (Traversable f, Num a) => FU f a -> f (a, a) -> f (a, a)Source

computes both the gradient of a non-scalar-to-scalar `hessianProduct'`

f wv`f`

at `w = `

and the product of the hessian `fst`

$ wv`H`

at `w`

with a vector `v = snd $ wv`

using "Pearlmutter's method". The outputs are returned wrapped in the same functor.

H v = (d/dr) grad_w (w + r v) | r = 0

Or in other words, we take the directional derivative of the gradient.

# Higher Order Gradients/Hessians (Sparse Forward)

# Derivatives (Forward Mode)

diff' :: Num a => UU a -> a -> (a, a)Source

The `d'UU`

function calculates the result and first derivative of scalar-to-scalar function by F`orward`

`AD`

d' sin == sin &&& cos d' f = f &&& d f

# Derivatives (Tower)

# Directional Derivatives (Forward Mode)

# Directional Derivatives (Tower)

# Taylor Series (Tower)

taylor :: Fractional a => UU a -> a -> a -> [a]Source

taylor0 :: Fractional a => UU a -> a -> a -> [a]Source

# Maclaurin Series (Tower)

maclaurin :: Fractional a => UU a -> a -> [a]Source

maclaurin0 :: Fractional a => UU a -> a -> [a]Source

# Monadic Combinators (Forward Mode)

diffM :: (Monad m, Num a) => UF m a -> a -> m aSource

The `dUM`

function calculates the first derivative of scalar-to-scalar monadic function by F`orward`

`AD`

diffM' :: (Monad m, Num a) => UF m a -> a -> m (a, a)Source

The `d'UM`

function calculates the result and first derivative of a scalar-to-scalar monadic function by F`orward`

`AD`

# Exposed Types

module Numeric.AD.Types

class Lifted t => Mode t whereSource

lift :: Num a => a -> t aSource

Embed a constant

(<+>) :: Num a => t a -> t a -> t aSource

Vector sum

(*^) :: Num a => a -> t a -> t aSource

Scalar-vector multiplication

(^*) :: Num a => t a -> a -> t aSource

Vector-scalar multiplication

(^/) :: Fractional a => t a -> a -> t aSource

Scalar division

'zero' = 'lift' 0