WordAlignment-0.1.0.0: Bigram word pair alignments.

Safe HaskellNone
LanguageHaskell2010

Linguistics.Bigram

Description

Map between Strings that represent characters and their Int-based representation.

NOTE filtering the scores list and creating a single bigram map takes about 70 seconds.

NOTE A single bigram map costs around 160 MByte ram. This includes the overhead for actually storing the bigrams once (creating pointers instead of multiple copied Bigram data structures.

Synopsis

Documentation

withDefault :: Double -> [ByteString] -> (Double, [ByteString]) Source

Try to read the first line to figure out if there is a default score there

type Lang = BTI Source

data Mapping Source

Constructors

Mapping 

Fields

bigrams :: !(Map Bigram Bigram)
 
lliid :: !(Map (Lang :!: Lang) Scores)
 

Instances

generateLookups :: Set BTI -> Double -> ByteString -> Mapping Source

Given a set of acceptable languages, a default score, and the lazy bytestring of scores, create the Mapping of languages and scores.