Copyright | (c) Melinae, 2014 Chad Scherrer, 2014 |
---|---|
License | MIT |
Maintainer | chad.scherrer@gmail.com |
Stability | experimental |
Portability | POSIX |
Safe Haskell | None |
Language | Haskell2010 |
This module gives an implementation of Bayesian linear regression, with the scale of the prior chosen by marginal likelihood.
The inputs for a Bayesian linear model are identical to those of a classical linear model, except that in addition to a design matrix and response, we must also specify a prior distribution on the weights and the noise. This leaves us with an open question of how these should be specified.
In his book Pattern Recognition and Machine Learning, Christopher Bishop provides details for an approach that simplifies the situation significantly, and allows for much faster inference. The structure of the linear model allows us to integrate the posterior over the weights, resulting in the marginal likelihood, expressed as a function of the prior precision and noise precision. This, in turn, can be easily optimized.
- data Fit
- fit :: ([(Double, Double)] -> (Double, Double)) -> Matrix Double -> Vector Double -> Fit
- design :: Fit -> Matrix Double
- response :: Fit -> Vector Double
- priorPrecision :: Fit -> Double
- noisePrecision :: Fit -> Double
- numEffectiveParameters :: Fit -> Double
- logEvidence :: Fit -> Double
- mapWeights :: Fit -> Vector Double
- hessian :: Fit -> Matrix Double
Documentation
:: ([(Double, Double)] -> (Double, Double)) | How to take the limit of the (α,β) sequence. A simple approach is, e.g., |
-> Matrix Double | The design matrix (each column is a feature) |
-> Vector Double | The response vector |
-> Fit |
fit lim x y
fits a Bayesian linear model to a design matrix x
and response vector y
. This is an iterative algorithm, resulting in a sequence (list) of (α,β) values. Here α is the prior precision, and β is the noise precision. The lim
function passed in is used to specify how the limit of this sequence should be computed.
priorPrecision :: Fit -> Double Source
The precision (inverse variance) of the prior distribution, determined by maximizing the marginal likelihood
noisePrecision :: Fit -> Double Source
The precision (inverse variance) of the noise
numEffectiveParameters :: Fit -> Double Source
The number of effective parameters in the model
logEvidence :: Fit -> Double Source
The log of the evidence, which is useful for model comparison (different features, same response)
mapWeights :: Fit -> Vector Double Source
The MAP (maximum a posteriori) values for the parameter weights