Safe Haskell	None
Language	Haskell2010

Goal.Geometry.Differential.GradientPursuit

Contents

Cauchy Sequences
Gradient Pursuit
- Algorithms
  - Defaults

Description

Gradient pursuit-based optimization on manifolds.

Synopsis

cauchyLimit :: ((c # x) -> (c # x) -> Double) -> Double -> [c # x] -> c # x
cauchySequence :: ((c # x) -> (c # x) -> Double) -> Double -> [c # x] -> [c # x]
vanillaGradient :: Manifold x => (c #* x) -> c # x
gradientStep :: Manifold x => Double -> (c # x) -> (c # x) -> c # x
data GradientPursuit
- = Classic
- | Momentum (Int -> Double)
- | Adam Double Double Double
gradientPursuitStep :: Manifold x => Double -> GradientPursuit -> Int -> (c # x) -> (c # x) -> [c # x] -> (c # x, [c # x])
gradientSequence :: Riemannian c x => ((c # x) -> c #* x) -> Double -> GradientPursuit -> (c # x) -> [c # x]
vanillaGradientSequence :: Manifold x => ((c # x) -> c #* x) -> Double -> GradientPursuit -> (c # x) -> [c # x]
gradientCircuit :: (Monad m, Manifold x) => Double -> GradientPursuit -> Circuit m (c # x, c # x) (c # x)
vanillaGradientCircuit :: (Monad m, Manifold x) => Double -> GradientPursuit -> Circuit m (c # x, c #* x) (c # x)
defaultMomentumPursuit :: Double -> GradientPursuit
defaultAdamPursuit :: GradientPursuit

Cauchy Sequences

cauchyLimit Source #

Arguments

:: ((c # x) -> (c # x) -> Double)	Distance (divergence) from previous to next
-> Double	Epsilon
-> [c # x]	Input sequence
-> c # x

Attempts to calculate the limit of a sequence by finding the iteration with a sufficiently small distance from its previous iteration.

cauchySequence Source #

Arguments

:: ((c # x) -> (c # x) -> Double)	Distance (divergence) from previous to next
-> Double	Epsilon
-> [c # x]	Input list
-> [c # x]	Truncated list

Attempts to calculate the limit of a sequence. Returns the list up to the limit.

Gradient Pursuit

vanillaGradient :: Manifold x => (c #* x) -> c # x Source #

Ignore the Riemannian metric, and convert a Point from a Dual space to its Primal space.

gradientStep Source #

Arguments

:: Manifold x
=> Double
-> (c # x)	Point
-> (c # x)	Tangent Vector
-> c # x	Stepped point

gradientStep takes a step size, a Point, a tangent vector at that point, and returns a Point with coordinates that have moved in the direction of the tangent vector.

Algorithms

data GradientPursuit Source #

An ADT reprenting three basic gradient descent algorithms.

Constructors

Classic
Momentum (Int -> Double)
Adam Double Double Double

gradientPursuitStep Source #

Arguments

:: Manifold x
=> Double	Learning Rate
-> GradientPursuit	Gradient pursuit algorithm
-> Int	Algorithm step
-> (c # x)	The point
-> (c # x)	The derivative
-> [c # x]	The velocities
-> (c # x, [c # x])	The updated point and velocities

A single step of a gradient pursuit algorithm.

gradientSequence Source #

Arguments

:: Riemannian c x
=> ((c # x) -> c #* x)	Differential calculator
-> Double	Step size
-> GradientPursuit	Gradient pursuit algorithm
-> (c # x)	The initial point
-> [c # x]	The gradient ascent

Gradient ascent based on the Riemannian metric.

vanillaGradientSequence Source #

Arguments

:: Manifold x
=> ((c # x) -> c #* x)	Differential calculator
-> Double	Step size
-> GradientPursuit	Gradient pursuit algorithm
-> (c # x)	The initial point
-> [c # x]	The gradient ascent

Gradient ascent which ignores the Riemannian metric.

gradientCircuit Source #

Arguments

:: (Monad m, Manifold x)
=> Double	Learning Rate
-> GradientPursuit	Gradient pursuit algorithm
-> Circuit m (c # x, c # x) (c # x)	(Point, Gradient) to Updated Point

A Circuit for gradient descent.

vanillaGradientCircuit Source #

Arguments

:: (Monad m, Manifold x)
=> Double	Learning Rate
-> GradientPursuit	Gradient pursuit algorithm
-> Circuit m (c # x, c #* x) (c # x)	(Point, Gradient) to Updated Point

A Circuit for gradient descent.

Defaults

defaultMomentumPursuit :: Double -> GradientPursuit Source #

A standard momentum schedule.

defaultAdamPursuit :: GradientPursuit Source #

Standard Adam parameters.