Numeric.SGD.Grad

A gradient is represented by an IntMap from gradient indices to values. Elements with no associated values in the gradient are assumed to have a 0 value assigned. Such elements are not interesting: when adding the gradient to the vector of parameters, only nonzero elements are taken into account.

Each value associated with a gradient position is a pair of positive and negative components. They are stored separately to ensure high accuracy of computation results. Besides, both positive and negative components are stored in a logarithmic domain.

Documentation

type Grad = IntMap LogSigned

Gradient with nonzero values stored in a logarithmic domain. Since values equal to zero have no impact on the update phase of the SGD method, it is more efficient to not to store those components in the gradient.

empty :: Grad

Empty gradient, i.e. with all elements set to 0.

add :: Grad -> Int -> Double -> Grad

Add normal-domain double to the gradient at the given position.

addL :: Grad -> Int -> LogSigned -> Grad

Add log-domain, singed number to the gradient at the given position.

fromList :: [(Int, Double)] -> Grad

Construct gradient from a list of (index, value) pairs. All values from the list are added at respective gradient positions.

fromLogList :: [(Int, LogSigned)] -> Grad

Construct gradient from a list of (index, signed, log-domain number) pairs. All values from the list are added at respective gradient positions.

toList :: Grad -> [(Int, Double)]

Collect gradient components with values in normal domain.

parUnions :: [Grad] -> Grad

Perform parallel unions operation on gradient list. Experimental version.