Maintainer Kiet Lam Safe-Infered

Math.LevenbergMarquardt

Description

The Levenberg-Marquardt algorithm is a minimization algorithm for functions expressed as a sum of squared errors

This can be used for curve-fitting, multidimensional function minimization, and neural networks training

Synopsis

# Documentation

Type that represents the function that can calculate the residues

Type that represents the function that can calculate the jacobian matrix of the residue with respect to each parameter

Arguments

 :: Function Multi-dimensional function that will return a vector of residues -> Jacobian The function that calculate the Jacobian matrix of each residue with respect to each parameter -> Vector Double The initial guess for the parameter -> Double Dampening constant (usually lambda in most literature) -> Double Dampening update value (usually beta in most literature) -> Double The precision desired -> Int The maximum iteration -> (Vector Double, Matrix Double) Returns the optimal parameter and the matrix path

Evolves the parameter x for f(x) = sum-square(e(x)) so that f(x) will be minimized, where:

f = real-valued error function, e(x) = {e1(x),e2(x),..,eN(x)}, where e1(x) = the residue at the vector x

NOTE: eN(x) is usually represented as (sample - hypothesis(x))

e.g.: In training neural networks, hypothesis(x) would be the network's output for a training set, and sample would be the expected output for that training set

NOTE: The dampening constant(lambda) should be set to 0.01 and the dampening update value (beta) should be set to be 10