A gradient is represented by an IntMap from gradient indices to values. Elements with no associated values in the gradient are assumed to have a 0 value assigned. Such elements are not interesting: when adding the gradient to the vector of parameters, only nonzero elements are taken into account.
Each value associated with a gradient position is a pair of positive and negative components. They are stored separately to ensure high accuracy of computation results. Besides, both positive and negative components are stored in a logarithmic domain.
Gradient with nonzero values stored in a logarithmic domain. Since values equal to zero have no impact on the update phase of the SGD method, it is more efficient to not to store those components in the gradient.
Add normal-domain double to the gradient at the given position.
Add log-domain, singed number to the gradient at the given position.
Construct gradient from a list of (index, value) pairs. All values from the list are added at respective gradient positions.
Construct gradient from a list of (index, signed, log-domain number) pairs. All values from the list are added at respective gradient positions.