Safe Haskell | None |
---|
The module provides first-order, linear-chain conditional random fields (CRFs) with position-wide constraints over label values.
- data Word a b = Word {}
- unknown :: Word a b -> Bool
- type Sent a b = [Word a b]
- data Prob a
- mkProb :: Ord a => [(a, Double)] -> Prob a
- type WordL a b = (Word a b, Prob b)
- type SentL a b = [WordL a b]
- data CRF a b = CRF {}
- train :: (Ord a, Ord b) => SgdArgs -> Bool -> IO [SentL a b] -> IO [SentL a b] -> (AVec Lb -> [(Xs, Ys)] -> [Feature]) -> IO (CRF a b)
- tag :: (Ord a, Ord b) => CRF a b -> Sent a b -> [b]
- tagK :: (Ord a, Ord b) => Int -> CRF a b -> Sent a b -> [[b]]
- hiddenFeats :: AVec Lb -> [(Xs, b)] -> [Feature]
- presentFeats :: [(Xs, Ys)] -> [Feature]
Data types
A Word is represented by a set of observations and a set of potential interpretation labels. When the set of potential labels is empty the word is considered to be unknown and the default potential set is used in its place.
unknown :: Word a b -> BoolSource
The word is considered to be unknown when the set of potential labels is empty.
A probability distribution defined over elements of type a. All elements not included in the map have probability equal to 0.
type WordL a b = (Word a b, Prob b)Source
A WordL is a labeled word, i.e. a word with probability distribution defined over labels. We assume that every label from the distribution domain is a member of the set of potential labels corresponding to the word. TODO: Ensure the assumption using the smart constructor.
CRF
A conditional random field model with additional codec used for data encoding.
Training
:: (Ord a, Ord b) | |
=> SgdArgs | Args for SGD |
-> Bool | Store dataset on a disk |
-> IO [SentL a b] | Training data |
-> IO [SentL a b] | Evaluation data |
-> (AVec Lb -> [(Xs, Ys)] -> [Feature]) | Feature selection |
-> IO (CRF a b) | Resulting model |
Train the CRF using the stochastic gradient descent method.
The resulting model will contain features extracted with
the user supplied extraction function.
You can use the functions provided by the Data.CRF.Chain1.Feature.Present
and Data.CRF.Chain1.Feature.Hidden modules for this purpose.
When the evaluation data IO
action is Just
, the iterative
training process will notify the user about the current accuracy
on the evaluation part every full iteration over the training part.
TODO: Accept custom r0 construction function.
Tagging
tag :: (Ord a, Ord b) => CRF a b -> Sent a b -> [b]Source
Determine the most probable label sequence within the context of the
given sentence using the model provided by the CRF
.
tagK :: (Ord a, Ord b) => Int -> CRF a b -> Sent a b -> [[b]]Source
Determine the most probable label sets of the given size (at maximum) for each position in the input sentence.