full-text-search-0.2.1.4: In-memory full text search engine

Data.SearchEngine.BM25F

Description

An implementation of BM25F ranking. See:

Synopsis

# The ranking function

score :: (Ix field, Bounded field, Ix feature, Bounded feature) => Context term field feature -> Doc term field feature -> [term] -> Float Source #

The BM25F score for a document for a given set of terms.

data Context term field feature Source #

Constructors

 Context FieldsnumDocsTotal :: !Int avgFieldLength :: field -> Float numDocsWithTerm :: term -> Int paramK1 :: !Float paramB :: field -> Float fieldWeight :: field -> Float featureWeight :: feature -> Float featureFunction :: feature -> FeatureFunction

Constructors

 LogarithmicFunction Float log (lambda_i + f_i) RationalFunction Float f_i / (lambda_i + f_i) SigmoidFunction Float Float 1 / (lambda + exp(-(lambda' * f_i))

data Doc term field feature Source #

Constructors

 Doc FieldsdocFieldLength :: field -> Int docFieldTermFrequency :: field -> term -> Int docFeatureValue :: feature -> Float

## Specialised variants

scoreTermsBulk :: forall field term feature. (Ix field, Bounded field) => Context term field feature -> Doc term field feature -> term -> (field -> Int) -> Float Source #

Most of the time we want to score several different documents for the same set of terms, but sometimes we want to score one document for many terms and in that case we can save a bit of work by doing it in bulk. It lets us calculate once and share things that depend only on the document, and not the term.

To take advantage of the sharing you must partially apply and name the per-doc score functon, e.g.

let score :: term -> (field -> Int) -> Float
score = BM25.bulkScorer ctx doc
in sum [ score t (\f -> counts ! (t, f)) | t <- ts ]

# Explaining the score

data Explanation field feature term Source #

A breakdown of the BM25F score, to explain somewhat how it relates to the inputs, and so you can compare the scores of different documents.

Constructors

 Explanation FieldsoverallScore :: FloatThe overall score is the sum of the termScores, positionScore and nonTermScoretermScores :: [(term, Float)]There is a score contribution from each query term. This is the score for the term across all fields in the document (but see termFieldScores).nonTermScores :: [(feature, Float)]The document can have an inate bonus score independent of the terms in the query. For example this might be a popularity score.termFieldScores :: [(term, [(field, Float)])]This does not contribute to the overallScore. It is an indication of how the termScores relates to per-field scores. Note however that the term score for all fields is not simply sum of the per-field scores. The point of the BM25F scoring function is that a linear combination of per-field scores is wrong, and BM25F does a more cunning non-linear combination.However, it is still useful as an indication to see scores for each field for a term, to see how the compare.

Instances

 Functor (Explanation field feature) Source # Methodsfmap :: (a -> b) -> Explanation field feature a -> Explanation field feature b #(<\$) :: a -> Explanation field feature b -> Explanation field feature a # (Show field, Show feature, Show term) => Show (Explanation field feature term) Source # MethodsshowsPrec :: Int -> Explanation field feature term -> ShowS #show :: Explanation field feature term -> String #showList :: [Explanation field feature term] -> ShowS #

explain :: (Ix field, Bounded field, Ix feature, Bounded feature) => Context term field feature -> Doc term field feature -> [term] -> Explanation field feature term Source #