Numeric.Backprop.Class

Provides the Backprop typeclass, a class for values that can be used for backpropagation.

This class replaces the old (version 0.1) API relying on Num.

Since: 0.2.0.0

Backpropagatable types

class Backprop a where

Class of values that can be backpropagated in general.

For instances of Num, these methods can be given by zeroNum, addNum, and oneNum. There are also generic options given in Numeric.Backprop.Class for functors, IsList instances, and Generic instances.

instance Backprop Double where
    zero = zeroNum
    add = addNum
    one = oneNum

If you leave the body of an instance declaration blank, GHC Generics will be used to derive instances if the type has a single constructor and each field is an instance of Backprop.

To ensure that backpropagation works in a sound way, should obey the laws:

identity

• add x (zero y) = x

• add (zero x) y = y

Also implies preservation of information, making zipWith (+) an illegal implementation for lists and vectors.

This is only expected to be true up to potential "extra zeroes" in x and y in the result.

commutativity

• add x y = add y x

associativity

• add x (add y z) = add (add x y) z

idempotence

• zero . zero = zero

• one . one = one

Note that not all values in the backpropagation process needs all of these methods: Only the "final result" needs one, for example. These are all grouped under one typeclass for convenience in defining instances, and also to talk about sensible laws. For fine-grained control, use the "explicit" versions of library functions (for example, in Numeric.Backprop.Explicit) instead of Backprop based ones.

This typeclass replaces the reliance on Num of the previous API (v0.1). Num is strictly more powerful than Backprop, and is a stronger constraint on types than is necessary for proper backpropagating. In particular, fromInteger is a problem for many types, preventing useful backpropagation for lists, variable-length vectors (like Data.Vector) and variable-size matrices from linear algebra libraries like hmatrix and accelerate.

Since: 0.2.0.0

Methods

zero :: a -> a

"Zero out" all components of a value. For scalar values, this should just be const 0. For vectors and matrices, this should set all components to zero, the additive identity.

Should be idempotent:

• zero . zero = zero

Should be as lazy as possible. This behavior is observed for all instances provided by this library.

See zeroNum for a pre-built definition for instances of Num and zeroFunctor for a definition for instances of Functor. If left blank, will automatically be genericZero, a pre-built definition for instances of Generic whose fields are all themselves instances of Backprop.

add :: a -> a -> a

Add together two values of a type. To combine contributions of gradients, so should be information-preserving:

• add x (zero y) = x

• add (zero x) y = y

Should be as strict as possible. This behavior is observed for all instances provided by this library.

See addNum for a pre-built definition for instances of Num and addFunctor for a definition for instances of Functor. If left blank, will automatically be genericAdd, a pre-built definition for instances of Generic with one constructor whose fields are all themselves instances of Backprop.

one :: a -> a

One all components of a value. For scalar values, this should just be const 1. For vectors and matrices, this should set all components to one, the multiplicative identity.

Should be idempotent:

• one . one = one

Should be as lazy as possible. This behavior is observed for all instances provided by this library.

See oneNum for a pre-built definition for instances of Num and oneFunctor for a definition for instances of Functor. If left blank, will automatically be genericOne, a pre-built definition for instances of Generic whose fields are all themselves instances of Backprop.

zero :: (Generic a, GZero (Rep a)) => a -> a

"Zero out" all components of a value. For scalar values, this should just be const 0. For vectors and matrices, this should set all components to zero, the additive identity.

Should be idempotent:

• zero . zero = zero

Should be as lazy as possible. This behavior is observed for all instances provided by this library.

See zeroNum for a pre-built definition for instances of Num and zeroFunctor for a definition for instances of Functor. If left blank, will automatically be genericZero, a pre-built definition for instances of Generic whose fields are all themselves instances of Backprop.

add :: (Generic a, GAdd (Rep a)) => a -> a -> a

Add together two values of a type. To combine contributions of gradients, so should be information-preserving:

• add x (zero y) = x

• add (zero x) y = y

Should be as strict as possible. This behavior is observed for all instances provided by this library.

See addNum for a pre-built definition for instances of Num and addFunctor for a definition for instances of Functor. If left blank, will automatically be genericAdd, a pre-built definition for instances of Generic with one constructor whose fields are all themselves instances of Backprop.

one :: (Generic a, GOne (Rep a)) => a -> a

One all components of a value. For scalar values, this should just be const 1. For vectors and matrices, this should set all components to one, the multiplicative identity.

Should be idempotent:

• one . one = one

Should be as lazy as possible. This behavior is observed for all instances provided by this library.

See oneNum for a pre-built definition for instances of Num and oneFunctor for a definition for instances of Functor. If left blank, will automatically be genericOne, a pre-built definition for instances of Generic whose fields are all themselves instances of Backprop.

Instances

Backprop Double

Methods

zero :: Double -> Double

add :: Double -> Double -> Double

one :: Double -> Double

Backprop Float

Methods

zero :: Float -> Float

add :: Float -> Float -> Float

one :: Float -> Float

Backprop Int

Methods

zero :: Int -> Int

add :: Int -> Int -> Int

one :: Int -> Int

Backprop Integer

Methods

zero :: Integer -> Integer

add :: Integer -> Integer -> Integer

one :: Integer -> Integer

Backprop ()

add is strict, but zero and one are lazy in their arguments.

Methods

zero :: () -> ()

add :: () -> () -> ()

one :: () -> ()

Backprop Void

Methods

zero :: Void -> Void

add :: Void -> Void -> Void

one :: Void -> Void

Backprop a => Backprop [a]

add assumes the shorter list has trailing zeroes, and the result has the length of the longest input.

Methods

zero :: [a] -> [a]

add :: [a] -> [a] -> [a]

one :: [a] -> [a]

Backprop a => Backprop (Maybe a)

Nothing is treated the same as Just 0. However, zero, add, and one preserve Nothing if all inputs are also Nothing.

Methods

zero :: Maybe a -> Maybe a

add :: Maybe a -> Maybe a -> Maybe a

one :: Maybe a -> Maybe a

Integral a => Backprop (Ratio a)

Methods

zero :: Ratio a -> Ratio a

add :: Ratio a -> Ratio a -> Ratio a

one :: Ratio a -> Ratio a

RealFloat a => Backprop (Complex a)

Methods

zero :: Complex a -> Complex a

add :: Complex a -> Complex a -> Complex a

one :: Complex a -> Complex a

Backprop a => Backprop (NonEmpty a)

add assumes the shorter list has trailing zeroes, and the result has the length of the longest input.

Methods

zero :: NonEmpty a -> NonEmpty a

add :: NonEmpty a -> NonEmpty a -> NonEmpty a

one :: NonEmpty a -> NonEmpty a

Backprop a => Backprop (Identity a)

Methods

zero :: Identity a -> Identity a

add :: Identity a -> Identity a -> Identity a

one :: Identity a -> Identity a

Backprop a => Backprop (IntMap a)

zero and one replace all current values, and add merges keys from both maps, adding in the case of double-occurrences.

Methods

zero :: IntMap a -> IntMap a

add :: IntMap a -> IntMap a -> IntMap a

one :: IntMap a -> IntMap a

Backprop a => Backprop (Seq a)

add assumes the shorter sequence has trailing zeroes, and the result has the length of the longest input.

Methods

zero :: Seq a -> Seq a

add :: Seq a -> Seq a -> Seq a

one :: Seq a -> Seq a

Backprop a => Backprop (I a)

Methods

zero :: I a -> I a

add :: I a -> I a -> I a

one :: I a -> I a

(Unbox a, Backprop a) => Backprop (Vector a)

Methods

zero :: Vector a -> Vector a

add :: Vector a -> Vector a -> Vector a

one :: Vector a -> Vector a

(Storable a, Backprop a) => Backprop (Vector a)

Methods

zero :: Vector a -> Vector a

add :: Vector a -> Vector a -> Vector a

one :: Vector a -> Vector a

(Prim a, Backprop a) => Backprop (Vector a)

Methods

zero :: Vector a -> Vector a

add :: Vector a -> Vector a -> Vector a

one :: Vector a -> Vector a

Backprop a => Backprop (Vector a)

Methods

zero :: Vector a -> Vector a

add :: Vector a -> Vector a -> Vector a

one :: Vector a -> Vector a

(Backprop a, Backprop b) => Backprop (a, b)

add is strict

Methods

zero :: (a, b) -> (a, b)

add :: (a, b) -> (a, b) -> (a, b)

one :: (a, b) -> (a, b)

Backprop (Proxy * a)

add is strict, but zero and one are lazy in their arguments.

Methods

zero :: Proxy * a -> Proxy * a

add :: Proxy * a -> Proxy * a -> Proxy * a

one :: Proxy * a -> Proxy * a

(Backprop a, Ord k) => Backprop (Map k a)

zero and one replace all current values, and add merges keys from both maps, adding in the case of double-occurrences.

Methods

zero :: Map k a -> Map k a

add :: Map k a -> Map k a -> Map k a

one :: Map k a -> Map k a

(Backprop a, Backprop b, Backprop c) => Backprop (a, b, c)

add is strict

Methods

zero :: (a, b, c) -> (a, b, c)

add :: (a, b, c) -> (a, b, c) -> (a, b, c)

one :: (a, b, c) -> (a, b, c)

ListC ((<$>) * Constraint Backprop ((<$>) * * f as)) => Backprop (Prod * f as)

Methods

zero :: Prod * f as -> Prod * f as

add :: Prod * f as -> Prod * f as -> Prod * f as

one :: Prod * f as -> Prod * f as

MaybeC ((<$>) * Constraint Backprop ((<$>) * * f a)) => Backprop (Option * f a)

Methods

zero :: Option * f a -> Option * f a

add :: Option * f a -> Option * f a -> Option * f a

one :: Option * f a -> Option * f a

(Backprop a, Backprop b, Backprop c, Backprop d) => Backprop (a, b, c, d)

add is strict

Methods

zero :: (a, b, c, d) -> (a, b, c, d)

add :: (a, b, c, d) -> (a, b, c, d) -> (a, b, c, d)

one :: (a, b, c, d) -> (a, b, c, d)

(Backprop a, Backprop b, Backprop c, Backprop d, Backprop e) => Backprop (a, b, c, d, e)

add is strict

Methods

zero :: (a, b, c, d, e) -> (a, b, c, d, e)

add :: (a, b, c, d, e) -> (a, b, c, d, e) -> (a, b, c, d, e)

one :: (a, b, c, d, e) -> (a, b, c, d, e)

Derived methods

zeroNum :: Num a => a -> a

zero for instances of Num.

Is lazy in its argument.

addNum :: Num a => a -> a -> a

add for instances of Num.

oneNum :: Num a => a -> a

one for instances of Num.

Is lazy in its argument.

zeroVec :: (Vector v a, Backprop a) => v a -> v a

zero for instances of Vector.

addVec :: (Vector v a, Backprop a) => v a -> v a -> v a

add for instances of Vector. Automatically pads the end of the shorter vector with zeroes.

oneVec :: (Vector v a, Backprop a) => v a -> v a

one for instances of Vector.

zeroFunctor :: (Functor f, Backprop a) => f a -> f a

zero for Functor instances.

addIsList :: (IsList a, Backprop (Item a)) => a -> a -> a

add for instances of IsList. Automatically pads the end of the "shorter" value with zeroes.

addAsList

Arguments

:: Backprop b

=> (a -> [b])

convert to list (should form isomorphism)

-> ([b] -> a)

convert from list (should form isomorphism)

-> a

-> a

-> a

add for types that are isomorphic to a list. Automatically pads the end of the "shorter" value with zeroes.

oneFunctor :: (Functor f, Backprop a) => f a -> f a

one for instances of Functor.

genericZero :: (Generic a, GZero (Rep a)) => a -> a

zero using GHC Generics; works if all fields are instances of Backprop.

genericAdd :: (Generic a, GAdd (Rep a)) => a -> a -> a

add using GHC Generics; works if all fields are instances of Backprop, but only for values with single constructors.

genericOne :: (Generic a, GOne (Rep a)) => a -> a

one using GHC Generics; works if all fields are instaces of Backprop.

Generics

class GZero f where

Helper class for automatically deriving zero using GHC Generics.

Minimal complete definition

gzero

Methods

gzero :: f t -> f t

Instances

GZero (V1 *)

Methods

gzero :: V1 * t -> V1 * t

GZero (U1 *)

Methods

gzero :: U1 * t -> U1 * t

Backprop a => GZero (K1 * i a)

Methods

gzero :: K1 * i a t -> K1 * i a t

(GZero f, GZero g) => GZero ((:+:) * f g)

Methods

gzero :: (* :+: f) g t -> (* :+: f) g t

(GZero f, GZero g) => GZero ((:*:) * f g)

Methods

gzero :: (* :*: f) g t -> (* :*: f) g t

GZero f => GZero (M1 * i c f)

Methods

gzero :: M1 * i c f t -> M1 * i c f t

GZero f => GZero ((:.:) * * f g)

Methods

gzero :: (* :.: *) f g t -> (* :.: *) f g t

class GAdd f where

Helper class for automatically deriving add using GHC Generics.

Minimal complete definition

gadd

Methods

gadd :: f t -> f t -> f t

Instances

GAdd (V1 *)

Methods

gadd :: V1 * t -> V1 * t -> V1 * t

GAdd (U1 *)

Methods

gadd :: U1 * t -> U1 * t -> U1 * t

Backprop a => GAdd (K1 * i a)

Methods

gadd :: K1 * i a t -> K1 * i a t -> K1 * i a t

(GAdd f, GAdd g) => GAdd ((:*:) * f g)

Methods

gadd :: (* :*: f) g t -> (* :*: f) g t -> (* :*: f) g t

GAdd f => GAdd (M1 * i c f)

Methods

gadd :: M1 * i c f t -> M1 * i c f t -> M1 * i c f t

GAdd f => GAdd ((:.:) * * f g)

Methods

gadd :: (* :.: *) f g t -> (* :.: *) f g t -> (* :.: *) f g t

class GOne f where

Helper class for automatically deriving one using GHC Generics.

Minimal complete definition

gone

Methods

gone :: f t -> f t

Instances

GOne (V1 *)

Methods

gone :: V1 * t -> V1 * t

GOne (U1 *)

Methods

gone :: U1 * t -> U1 * t

Backprop a => GOne (K1 * i a)

Methods

gone :: K1 * i a t -> K1 * i a t

(GOne f, GOne g) => GOne ((:+:) * f g)

Methods

gone :: (* :+: f) g t -> (* :+: f) g t

(GOne f, GOne g) => GOne ((:*:) * f g)

Methods

gone :: (* :*: f) g t -> (* :*: f) g t

GOne f => GOne (M1 * i c f)

Methods

gone :: M1 * i c f t -> M1 * i c f t

GOne f => GOne ((:.:) * * f g)

Methods

gone :: (* :.: *) f g t -> (* :.: *) f g t