Numeric.AD.Mode.Mixed

Mixed-Mode Automatic Differentiation.

Each combinator exported from this module chooses an appropriate AD mode. The following basic operations are supported, modified as appropriate by the suffixes below:

• grad computes the gradient (partial derivatives) of a function at a point
• jacobian computes the Jacobian matrix of a function at a point
• diff computes the derivative of a function at a point
• du computes a directional derivative of a function at a point
• hessian compute the Hessian matrix (matrix of second partial derivatives) of a function at a point

The suffixes have the following meanings:

• ' -- also return the answer
• With lets the user supply a function to blend the input with the output
• F is a version of the base function lifted to return a Traversable (or Functor) result
• s means the function returns all higher derivatives in a list or f-branching Stream
• T means the result is transposed with respect to the traditional formulation.
• 0 means that the resulting derivative list is padded with 0s at the end.

Gradients (Reverse Mode)

grad :: (Traversable f, Num a) => FU f a -> f a -> f a

The grad function calculates the gradient of a non-scalar-to-scalar function with Reverse AD in a single pass.

grad' :: (Traversable f, Num a) => FU f a -> f a -> (a, f a)

The grad' function calculates the result and gradient of a non-scalar-to-scalar function with Reverse AD in a single pass.

gradWith :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> f b

grad g f function calculates the gradient of a non-scalar-to-scalar function f with reverse-mode AD in a single pass. The gradient is combined element-wise with the argument using the function g.

 grad == gradWith (\_ dx -> dx)
 id == gradWith const

gradWith' :: (Traversable f, Num a) => (a -> a -> b) -> FU f a -> f a -> (a, f b)

grad' g f calculates the result and gradient of a non-scalar-to-scalar function f with Reverse AD in a single pass the gradient is combined element-wise with the argument using the function g.

 grad' == gradWith' (\_ dx -> dx)

Higher Order Gradients (Sparse-on-Reverse)

grads :: (Traversable f, Num a) => FU f a -> f a -> Cofree f a

Jacobians (Sparse or Reverse)

jacobian :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (f a)

Calculate the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.

If you know the relative number of inputs and outputs, consider Numeric.AD.Reverse.jacobian or Nuneric.AD.Sparse.jacobian.

jacobian' :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (a, f a)

Calculate both the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward- and reverse- mode AD based on the relative, based on the number of inputs

If you know the relative number of inputs and outputs, consider Numeric.AD.Reverse.jacobian' or Nuneric.AD.Sparse.jacobian'.

jacobianWith :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (f b)

jacobianWith g f calculates the Jacobian of a non-scalar-to-non-scalar function, automatically choosing between forward and reverse mode AD based on the number of inputs and outputs.

The resulting Jacobian matrix is then recombined element-wise with the input using g.

If you know the relative number of inputs and outputs, consider Numeric.AD.Reverse.jacobianWith or Nuneric.AD.Sparse.jacobianWith.

jacobianWith' :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> g (a, f b)

jacobianWith' g f calculates the answer and Jacobian of a non-scalar-to-non-scalar function, automatically choosing between sparse and reverse mode AD based on the number of inputs and outputs.

The resulting Jacobian matrix is then recombined element-wise with the input using g.

If you know the relative number of inputs and outputs, consider Numeric.AD.Reverse.jacobianWith' or Nuneric.AD.Sparse.jacobianWith'.

Higher Order Jacobian (Sparse-on-Reverse)

jacobians :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (Cofree f a)

Transposed Jacobians (Forward Mode)

jacobianT :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> f (g a)

A fast, simple transposed Jacobian computed with forward-mode AD.

jacobianWithT :: (Traversable f, Functor g, Num a) => (a -> a -> b) -> FF f g a -> f a -> f (g b)

A fast, simple transposed Jacobian computed with forward-mode AD.

Hessian (Sparse-On-Reverse)

hessian :: (Traversable f, Num a) => FU f a -> f a -> f (f a)

Compute the Hessian via the Jacobian of the gradient. gradient is computed in reverse mode and then the Jacobian is computed in sparse (forward) mode.

hessian' :: (Traversable f, Num a) => FU f a -> f a -> (a, f (a, f a))

Hessian Tensors (Sparse or Sparse-On-Reverse)

hessianF :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (f (f a))

Compute the order 3 Hessian tensor on a non-scalar-to-non-scalar function using Sparse or Sparse-on-Reverse

Hessian Tensors (Sparse)

hessianF' :: (Traversable f, Functor g, Num a) => FF f g a -> f a -> g (a, f (a, f a))

Hessian Vector Products (Forward-On-Reverse)

hessianProduct :: (Traversable f, Num a) => FU f a -> f (a, a) -> f a

hessianProduct f wv computes the product of the hessian H of a non-scalar-to-scalar function f at w = fst $ wv with a vector v = snd $ wv using "Pearlmutter's method" from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.6143, which states:

 H v = (d/dr) grad_w (w + r v) | r = 0

Or in other words, we take the directional derivative of the gradient. The gradient is calculated in reverse mode, then the directional derivative is calculated in forward mode.

hessianProduct' :: (Traversable f, Num a) => FU f a -> f (a, a) -> f (a, a)

hessianProduct' f wv computes both the gradient of a non-scalar-to-scalar f at w = fst $ wv and the product of the hessian H at w with a vector v = snd $ wv using "Pearlmutter's method". The outputs are returned wrapped in the same functor.

 H v = (d/dr) grad_w (w + r v) | r = 0

Or in other words, we return the gradient and the directional derivative of the gradient. The gradient is calculated in reverse mode, then the directional derivative is calculated in forward mode.

Derivatives (Forward Mode)

diff :: Num a => UU a -> a -> a

The diff function calculates the first derivative of a scalar-to-scalar function by forward-mode AD

 diff sin == cos

diffF :: (Functor f, Num a) => UF f a -> a -> f a

The diffF function calculates the first derivative of scalar-to-nonscalar function by Forward AD

diff' :: Num a => UU a -> a -> (a, a)

The d'UU function calculates the result and first derivative of scalar-to-scalar function by Forward AD

 d' sin == sin &&& cos
 d' f = f &&& d f

diffF' :: (Functor f, Num a) => UF f a -> a -> f (a, a)

The diffF' function calculates the result and first derivative of a scalar-to-non-scalar function by Forward AD

Derivatives (Tower)

diffs :: Num a => UU a -> a -> [a]

diffsF :: (Functor f, Num a) => UF f a -> a -> f [a]

diffs0 :: Num a => UU a -> a -> [a]

diffs0F :: (Functor f, Num a) => UF f a -> a -> f [a]

Directional Derivatives (Forward Mode)

du :: (Functor f, Num a) => FU f a -> f (a, a) -> a

du' :: (Functor f, Num a) => FU f a -> f (a, a) -> (a, a)

duF :: (Functor f, Functor g, Num a) => FF f g a -> f (a, a) -> g a

duF' :: (Functor f, Functor g, Num a) => FF f g a -> f (a, a) -> g (a, a)

Directional Derivatives (Tower)

dus :: (Functor f, Num a) => FU f a -> f [a] -> [a]

dus0 :: (Functor f, Num a) => FU f a -> f [a] -> [a]

dusF :: (Functor f, Functor g, Num a) => FF f g a -> f [a] -> g [a]

dus0F :: (Functor f, Functor g, Num a) => FF f g a -> f [a] -> g [a]

Taylor Series (Tower)

taylor :: Fractional a => UU a -> a -> a -> [a]

taylor0 :: Fractional a => UU a -> a -> a -> [a]

Maclaurin Series (Tower)

maclaurin :: Fractional a => UU a -> a -> [a]

maclaurin0 :: Fractional a => UU a -> a -> [a]

Unsafe Variadic Grad

vgrad :: Grad i o o' a => i -> o

vgrad' :: Grad i o o' a => i -> o'

vgrads :: Grads i o a => i -> o

Exposed Types

module Numeric.AD.Types

class Lifted t => Mode t where

Methods

lift :: Num a => a -> t a

Embed a constant

(<+>) :: Num a => t a -> t a -> t a

Vector sum

(*^) :: Num a => a -> t a -> t a

Scalar-vector multiplication

(^*) :: Num a => t a -> a -> t a

Vector-scalar multiplication

(^/) :: Fractional a => t a -> a -> t a

Scalar division

zero :: Num a => t a

 'zero' = 'lift' 0

Instances

Mode Id

Lifted Forward => Mode Forward

Lifted Tower => Mode Tower

Lifted Reverse => Mode Reverse

Lifted Sparse => Mode Sparse

Mode f => Mode (AD f)

(Traversable f, Lifted (Dense f)) => Mode (Dense f)

(Mode f, Mode g) => Mode (ComposeMode f g)

class Num a => Grad i o o' a | i -> a o o', o -> a i o', o' -> a i o

Instances

Num a => Grad (AD Reverse a) [a] (a, [a]) a

Grad i o o' a => Grad (AD Reverse a -> i) (a -> o) (a -> o') a

class Num a => Grads i o a | i -> a o, o -> a i

Instances

Grads i o a => Grads (AD Sparse a -> i) (a -> o) a

Num a => Grads (AD Sparse a) (Cofree [] a) a