colada-0.4.3: Colada implements incremental word class class induction using online LDA

Safe Haskell	None

Colada.WordClass

Contents

Running the sampler
Extracting information
Class and word prediction
Data types and associated lenses

Description

Word Class induction with LDA

This module provides function which implement word class induction using the generic algorithm implemented in Colada.LDA.

You can access and set options in the Options record using lenses. Example:

  import Data.Label
  let options =   set passes 5 
                . set beta 0.01 
                . set topicNum 100 
                $ defaultOptions
  in run options sentences

Synopsis

Running the sampler

learn :: Options -> [Sentence] -> (WordClass, [Vector D])Source

learn options xs runs the LDA Gibbs sampler for word classes with options on sentences xs, and returns the resulting model together progressive class the assignments

defaultOptions :: Options Source

Extracting information

summary :: WordClass -> Text Source

summary m returns a textual summary of word classes found in model m

summarize :: Bool -> WordClass -> Text Source

wordTypeClasses :: WordClass -> Map Text (IntMap Double)Source

wordTypeClasses m returns a Map from word types to unnormalized distributions over word classes

Class and word prediction

label :: Bool -> WordClass -> Sentence -> Vector (Vector Double)Source

label m s returns for each word in sentences s, unnormalized probabilities of word classes.

predict :: WordClass -> Sentence -> [Vector (Double, Text)]Source

predict m s returns for each word in sentence s, unnormalized probabilities of words given predicted word class.

Data types and associated lenses

data WordClass Source

Container for the Word Class model

Instances

Generic WordClass
Serialize WordClass

ldaModel :: forall (~>). Arrow ~> => Lens ~> WordClass Finalized Source

LDA model

wordTypeTable :: forall (~>). Arrow ~> => Lens ~> WordClass (AtomTable (Vector Char))Source

Word type string to atom and vice versa conversion tables

featureTable :: forall (~>). Arrow ~> => Lens ~> WordClass (AtomTable (Vector Char))Source

Feature string to atom and vice versa conversion tables

options :: forall (~>). Arrow ~> => Lens ~> WordClass Options Source

Options for Gibbs sampling

Instances

Generic Finalized
Serialize Finalized

docTopics :: Finalized -> Table2D

Document topic counts

wordTopics :: Finalized -> Table2D

Word topic counts

topics :: Finalized -> Table1D

Topics counts

topicDocs :: Finalized -> Table2D

Inverse document-topic counts

topicWords :: Finalized -> Table2D

Inverse word-topic counts

data Options Source

Instances

Eq Options
Data Options
Show Options
Typeable Options
Generic Options
Serialize Options

featIds :: forall (~>). Arrow ~> => Lens ~> Options [Int]Source

Feature ids

topicNum :: forall (~>). Arrow ~> => Lens ~> Options Int Source

Number of topics K

alphasum :: forall (~>). Arrow ~> => Lens ~> Options Double Source

Dirichlet parameter alpha*K which controls topic sparseness

beta :: forall (~>). Arrow ~> => Lens ~> Options Double Source

Dirichlet parameter beta which controls word sparseness

passes :: forall (~>). Arrow ~> => Lens ~> Options Int Source

Number of sampling passes per batch

repeats :: forall (~>). Arrow ~> => Lens ~> Options Int Source

Number of repeats per sentences

batchSize :: forall (~>). Arrow ~> => Lens ~> Options Int Source

Number of sentences per batch

seed :: forall (~>). Arrow ~> => Lens ~> Options Word32 Source

Random seed for the sampler

topn :: forall (~>). Arrow ~> => Lens ~> Options Int Source

Number of most probable words to return

initSize :: forall (~>). Arrow ~> => Lens ~> Options Int Source

initPasses :: forall (~>). Arrow ~> => Lens ~> Options Int Source

exponent :: forall (~>). Arrow ~> => Lens ~> Options (Maybe Double)Source

progressive :: forall (~>). Arrow ~> => Lens ~> Options Bool Source

lambda :: forall (~>). Arrow ~> => Lens ~> Options Double Source