Portability GHC only experimental ekmett@gmail.com

Description

Mixed-Mode Automatic Differentiation.

Each combinator exported from this module chooses an appropriate AD mode. The following basic operations are supported, modified as appropriate by the suffixes below:

• `grad` computes the gradient (partial derivatives) of a function at a point
• `jacobian` computes the Jacobian matrix of a function at a point
• `diff` computes the derivative of a function at a point
• `du` computes a directional derivative of a function at a point
• `hessian` compute the Hessian matrix (matrix of second partial derivatives) of a function at a point

The suffixes have the following meanings:

• `'` -- also return the answer
• `With` lets the user supply a function to blend the input with the output
• `F` is a version of the base function lifted to return a `Traversable` (or `Functor`) result
• `s` means the function returns all higher derivatives in a list or f-branching `Stream`
• `T` means the result is transposed with respect to the traditional formulation.
• `0` means that the resulting derivative list is padded with 0s at the end.

Synopsis

grad :: (Traversable f, Num a) => FU f a -> f a -> f aSource

The `grad` function calculates the gradient of a non-scalar-to-scalar function with `Reverse` AD in a single pass.

grad' :: (Traversable f, Num a) => FU f a -> f a -> (a, f a)Source

The `grad'` function calculates the result and gradient of a non-scalar-to-scalar function with `Reverse` AD in a single pass.

gradWith :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> f bSource

`grad g f` function calculates the gradient of a non-scalar-to-scalar function `f` with reverse-mode AD in a single pass. The gradient is combined element-wise with the argument using the function `g`.

``` grad == gradWith (\_ dx -> dx)
```

gradWith' :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> (a, f b)Source

`grad' g f` calculates the result and gradient of a non-scalar-to-scalar function `f` with `Reverse` AD in a single pass the gradient is combined element-wise with the argument using the function `g`.

``` grad' == gradWith' (\_ dx -> dx)
```

grads :: (Traversable f, Num a) => FU f a -> f a -> Stream f aSource

# Jacobians (Sparse or Reverse)

jacobian :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (f a)Source

Calculate the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.

If you know the relative number of inputs and outputs, consider `Numeric.AD.Reverse.jacobian` or `Nuneric.AD.Sparse.jacobian`.

jacobian' :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (a, f a)Source

Calculate both the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward- and reverse- mode AD based on the relative, based on the number of inputs

If you know the relative number of inputs and outputs, consider `Numeric.AD.Reverse.jacobian'` or `Nuneric.AD.Sparse.jacobian'`.

jacobianWith :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)Source

`jacobianWith g f` calculates the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.

The resulting Jacobian matrix is then recombined element-wise with the input using `g`.

If you know the relative number of inputs and outputs, consider `Numeric.AD.Reverse.jacobianWith` or `Nuneric.AD.Sparse.jacobianWith`.

jacobianWith' :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)Source

`jacobianWith' g f` calculates the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between sparse and reverse mode AD based on the number of inputs and outputs.

The resulting Jacobian matrix is then recombined element-wise with the input using `g`.

If you know the relative number of inputs and outputs, consider `Numeric.AD.Reverse.jacobianWith'` or `Nuneric.AD.Sparse.jacobianWith'`.

# Higher Order Jacobian (Sparse-on-Reverse)

jacobians :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (Stream f a)Source

# Transposed Jacobians (Forward Mode)

jacobianT :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> f (g a)Source

A fast, simple transposed Jacobian computed with forward-mode AD.

jacobianWithT :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> f (g b)Source

A fast, simple transposed Jacobian computed with forward-mode AD.

# Hessian (Sparse-On-Reverse)

hessian :: (Traversable f, Num a) => FU f a -> f a -> f (f a)Source

Compute the Hessian via the Jacobian of the gradient. gradient is computed in reverse mode and then the Jacobian is computed in sparse (forward) mode.

hessian' :: (Traversable f, Num a) => FU f a -> f a -> (a, f (a, f a))Source

# Hessian Tensors (Sparse or Sparse-On-Reverse)

hessianF :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (f (f a))Source

Compute the order 3 Hessian tensor on a non-scalar-to-non-scalar function using Sparse or Sparse-on-Reverse

# Hessian Tensors (Sparse)

hessianF' :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (a, f (a, f a))Source

# Hessian Vector Products (Forward-On-Reverse)

hessianProduct :: (Traversable f, Num a) => FU f a -> f (a, a) -> f aSource

`hessianProduct f wv` computes the product of the hessian `H` of a non-scalar-to-scalar function `f` at `w = fst \$ wv` with a vector `v = snd \$ wv` using "Pearlmutter's method" from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.6143, which states:

``` H v = (d/dr) grad_w (w + r v) | r = 0
```

Or in other words, we take the directional derivative of the gradient. The gradient is calculated in reverse mode, then the directional derivative is calculated in forward mode.

hessianProduct' :: (Traversable f, Num a) => FU f a -> f (a, a) -> f (a, a)Source

`hessianProduct' f wv` computes both the gradient of a non-scalar-to-scalar `f` at `w = fst \$ wv` and the product of the hessian `H` at `w` with a vector `v = snd \$ wv` using "Pearlmutter's method". The outputs are returned wrapped in the same functor.

``` H v = (d/dr) grad_w (w + r v) | r = 0
```

Or in other words, we return the gradient and the directional derivative of the gradient. The gradient is calculated in reverse mode, then the directional derivative is calculated in forward mode.

# Derivatives (Forward Mode)

diff :: Num a => UU a -> a -> aSource

The `diff` function calculates the first derivative of a scalar-to-scalar function by forward-mode `AD`

``` diff sin == cos
```

diffF :: (Functor f, Num a) => UF f a -> a -> f aSource

The `diffF` function calculates the first derivative of scalar-to-nonscalar function by F`orward` `AD`

diff' :: Num a => UU a -> a -> (a, a)Source

The `d'UU` function calculates the result and first derivative of scalar-to-scalar function by F`orward` `AD`

``` d' sin == sin &&& cos
d' f = f &&& d f
```

diffF' :: (Functor f, Num a) => UF f a -> a -> f (a, a)Source

The `diffF'` function calculates the result and first derivative of a scalar-to-non-scalar function by F`orward` `AD`

# Derivatives (Tower)

diffs :: Num a => UU a -> a -> [a]Source

diffsF :: (Functor f, Num a) => UF f a -> a -> f [a]Source

diffs0 :: Num a => UU a -> a -> [a]Source

diffs0F :: (Functor f, Num a) => UF f a -> a -> f [a]Source

# Directional Derivatives (Forward Mode)

du :: (Functor f, Num a) => FU f a -> f (a, a) -> aSource

du' :: (Functor f, Num a) => FU f a -> f (a, a) -> (a, a)Source

duF :: (Functor f, Functor g, Num a) => FF f g a -> f (a, a) -> g aSource

duF' :: (Functor f, Functor g, Num a) => FF f g a -> f (a, a) -> g (a, a)Source

# Directional Derivatives (Tower)

dus :: (Functor f, Num a) => FU f a -> f [a] -> [a]Source

dus0 :: (Functor f, Num a) => FU f a -> f [a] -> [a]Source

dusF :: (Functor f, Functor g, Num a) => FF f g a -> f [a] -> g [a]Source

dus0F :: (Functor f, Functor g, Num a) => FF f g a -> f [a] -> g [a]Source

# Taylor Series (Tower)

taylor :: Fractional a => UU a -> a -> a -> [a]Source

taylor0 :: Fractional a => UU a -> a -> a -> [a]Source

# Maclaurin Series (Tower)

maclaurin :: Fractional a => UU a -> a -> [a]Source

maclaurin0 :: Fractional a => UU a -> a -> [a]Source

# Exposed Types

class Lifted t => Mode t whereSource

Methods

lift :: Num a => a -> t aSource

Embed a constant

(<+>) :: Num a => t a -> t a -> t aSource

Vector sum

(*^) :: Num a => a -> t a -> t aSource

Scalar-vector multiplication

(^*) :: Num a => t a -> a -> t aSource

Vector-scalar multiplication

(^/) :: Fractional a => t a -> a -> t aSource

Scalar division

zero :: Num a => t aSource

``` 'zero' = 'lift' 0
```

Instances

 Mode Id Lifted Forward => Mode Forward Lifted Tower => Mode Tower Lifted Reverse => Mode Reverse Lifted Sparse => Mode Sparse Mode f => Mode (AD f) (Traversable f, Lifted (Dense f)) => Mode (Dense f) (Mode f, Mode g) => Mode (ComposeMode f g)

class Num a => Grad i o o' a | i -> a o o', o -> a i o', o' -> a i oSource

Instances

 Num a => Grad (AD Reverse a) [a] (a, [a]) a Grad i o o' a => Grad (AD Reverse a -> i) (a -> o) (a -> o') a

class Num a => Grads i o a | i -> a o, o -> a iSource

Instances