MachineLearning.Regression

Documentation

class Model a where

Minimal complete definition

hypothesis, cost, gradient

Methods

hypothesis :: a -> Matrix -> Vector -> Vector

Hypothesis function, a.k.a. score function (for lassifition problem) Takes X (m x n) and theta (n x 1), returns y (m x 1).

cost :: a -> Regularization -> Matrix -> Vector -> Vector -> R

Cost function J(Theta), a.k.a. loss function. It takes regularizarion parameter, matrix X (m x n), vector y (m x 1) and vector theta (n x 1).

gradient :: a -> Regularization -> Matrix -> Vector -> Vector -> Vector

Gradient function. It takes regularizarion parameter, X (m x n), y (m x 1) and theta (n x 1). Returns vector of gradients (n x 1).

Instances

Model LogisticModel

Methods

hypothesis :: LogisticModel -> Matrix -> Vector -> Vector

cost :: LogisticModel -> Regularization -> Matrix -> Vector -> Vector -> R

gradient :: LogisticModel -> Regularization -> Matrix -> Vector -> Vector -> Vector

Model LeastSquaresModel

Methods

hypothesis :: LeastSquaresModel -> Matrix -> Vector -> Vector

cost :: LeastSquaresModel -> Regularization -> Matrix -> Vector -> Vector -> R

gradient :: LeastSquaresModel -> Regularization -> Matrix -> Vector -> Vector -> Vector

Model NeuralNetworkModel

Methods

hypothesis :: NeuralNetworkModel -> Matrix -> Vector -> Vector

cost :: NeuralNetworkModel -> Regularization -> Matrix -> Vector -> Vector -> R

gradient :: NeuralNetworkModel -> Regularization -> Matrix -> Vector -> Vector -> Vector

Classifier a => Model (MultiClassModel a)

Methods

hypothesis :: MultiClassModel a -> Matrix -> Vector -> Vector

cost :: MultiClassModel a -> Regularization -> Matrix -> Vector -> Vector -> R

gradient :: MultiClassModel a -> Regularization -> Matrix -> Vector -> Vector -> Vector

data LeastSquaresModel

Constructors

LeastSquares

Instances

Model LeastSquaresModel

Methods

hypothesis :: LeastSquaresModel -> Matrix -> Vector -> Vector

cost :: LeastSquaresModel -> Regularization -> Matrix -> Vector -> Vector -> R

gradient :: LeastSquaresModel -> Regularization -> Matrix -> Vector -> Vector -> Vector

data MinimizeMethod

Constructors

GradientDescent R

Gradient descent, takes alpha. Requires feature normalization.

MinibatchGradientDescent Int Int R

Minibacth Gradietn Descent, takes seed, batch size and alpha

ConjugateGradientFR R R

Fletcher-Reeves conjugate gradient algorithm, takes size of first trial step (0.1 is fine) and tol (0.1 is fine).

ConjugateGradientPR R R

Polak-Ribiere conjugate gradient algorithm. takes size of first trial step (0.1 is fine) and tol (0.1 is fine).

BFGS2 R R

Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm, takes size of first trial step (0.1 is fine) and tol (0.1 is fine).

minimize

Arguments

:: Model a

=> MinimizeMethod

-> a

model (Least Squares, Logistic Regression etc)

-> R

epsilon, desired precision of the solution

-> Int

maximum number of iterations allowed

-> Regularization

regularization parameter

-> Matrix

X

-> Vector

y

-> Vector

initial solution, theta

-> (Vector, Matrix)

solution vector and optimization path

Returns solution vector (theta) and optimization path. Optimization path's row format: [iter number, cost function value, theta values...]

normalEquation :: Matrix -> Vector -> Vector

Normal equation using inverse, does not require feature normalization It takes X and y, returns theta.

normalEquation_p :: Matrix -> Vector -> Vector

Normal equation using pseudo inverse, requires feature normalization It takes X and y, returns theta.

data Regularization

Regularization

Constructors

RegNone

No regularization

L2 R

L2 Regularization