Maintainer | Kiet Lam <ktklam9@gmail.com> |
---|

This module represents ways to calculate the gradient vector of the weights of the neural network

Backpropagation should always be preferred over the Numerical Gradient method

# Documentation

backpropagation :: GradientFunctionSource

Calculate the analytical gradient of the weights of the network by using backpropagation

:: Double | The epsilon |

-> GradientFunction | Returns a gradient function that calculates the numerical gradients of the weights |

NOTE: This should only be used as a last resort if for some reason (bugs?) the backpropagation algorithm does not give you good gradients

The numerical algorithm requires two forward propagations, while the backpropagation algorithm only requires one, so this is more costly

Also, analytical gradients almost always perform better than numerical gradients

User must provide an epsilon value. Make sure to use a very small value for the epsilon for more accurate gradients