Numeric.SGD

Stochastic gradient descent implementation using mutable vectors for efficient update of the parameters vector. A user is provided with the immutable vector of parameters so he is able to compute the gradient outside of the IO monad. Currently only the Gaussian priors are implemented.

This is a preliminary version of the SGD library and API may change in future versions.

Documentation

data SgdArgs

SGD parameters controlling the learning process.

Constructors

SgdArgs

Fields

batchSize :: Int

Size of the batch

regVar :: Double

Regularization variance

iterNum :: Double

Number of iterations

gain0 :: Double

Initial gain parameter

tau :: Double

After how many iterations over the entire dataset the gain parameter is halved

sgdArgsDefault :: SgdArgs

Default SGD parameter values.

type Para = Vector Double

Vector of parameters.

sgd

Arguments

:: SgdArgs

SGD parameter values

-> (Para -> Int -> IO ())

Notification run every update

-> (Para -> x -> Grad)

Gradient for dataset element

-> Dataset x

Dataset

-> Para

Starting point

-> IO Para

SGD result

A stochastic gradient descent method. A notification function can be used to provide user with information about the progress of the learning.

module Numeric.SGD.Grad

module Numeric.SGD.Dataset