MachineLearning.NeuralNetwork.WeightInitialization

Various Weight Initialization algorithms.

Documentation

nguyen :: RandomGen g => (Int, Int) -> Rand g (Matrix, Matrix)

Weight Initialization Algorithm discussed in Nguyen at al.: https://web.stanford.edu/class/ee373b/nninitialization.pdf Nguyen, Derrick, Widrow, B. Improving the learning speed of 2-layer neural networks by choosing initial values of adaptive weights. In Proc. IJCNN, 1990; 3: 21-26.

he :: RandomGen g => (Int, Int) -> Rand g (Matrix, Matrix)

Weight Initialization Algorithm discussed in He at al.: https://arxiv.org/abs/1502.01852 Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.