AI.Calculation.Gradients

This module represents ways to calculate the gradient vector of the weights of the neural network

Backpropagation should always be preferred over the Numerical Gradient method

Documentation

backpropagation :: GradientFunction

Calculate the analytical gradient of the weights of the network by using backpropagation

numericalGradients

Arguments

:: Double

The epsilon

-> GradientFunction

Returns a gradient function that calculates the numerical gradients of the weights

NOTE: This should only be used as a last resort if for some reason (bugs?) the backpropagation algorithm does not give you good gradients

The numerical algorithm requires two forward propagations, while the backpropagation algorithm only requires one, so this is more costly

Also, analytical gradients almost always perform better than numerical gradients

User must provide an epsilon value. Make sure to use a very small value for the epsilon for more accurate gradients