PGF

This module is an Application Programming Interface to load and interpret grammars compiled in Portable Grammar Format (PGF). The PGF format is produced as a final output from the GF compiler. The API is meant to be used for embedding GF grammars in Haskell programs

PGF

data PGF

An abstract data type representing multilingual grammar in Portable Grammar Format.

readPGF :: FilePath -> IO PGF

Reads file in Portable Grammar Format and produces PGF structure. The file is usually produced with:

 $ gf -make <grammar file name>

Identifiers

data CId

An abstract data type that represents identifiers for functions and categories in PGF.

Instances

Eq CId

Ord CId

Read CId

Show CId

mkCId :: String -> CId

Creates a new identifier from String

wildCId :: CId

showCId :: CId -> String

Renders the identifier as String

readCId :: String -> Maybe CId

Reads an identifier from String. The function returns Nothing if the string is not valid identifier.

ppCId :: CId -> Doc

pIdent :: ReadP String

utf8CId :: ByteString -> CId

Creates an identifier from a UTF-8-encoded ByteString

Languages

type Language = CId

This is just a CId with the language name. A language name is the identifier that you write in the top concrete or abstract module in GF after the concrete/abstract keyword. Example:

 abstract Lang = ...
 concrete LangEng of Lang = ...

showLanguage :: Language -> String

readLanguage :: String -> Maybe Language

languages :: PGF -> [Language]

List of all languages available in the given grammar.

abstractName :: PGF -> Language

The abstract language name is the name of the top-level abstract module

languageCode :: PGF -> Language -> Maybe String

Gets the RFC 4646 language tag of the language which the given concrete syntax implements, if this is listed in the source grammar. Example language tags include "en" for English, and "en-UK" for British English.

Types

data Type

To read a type from a String, use readType.

Instances

Eq Type

Ord Type

Show Type

type Hypo = (BindType, CId, Type)

Hypo represents a hypothesis in a type i.e. in the type A -> B, A is the hypothesis

showType :: [CId] -> Type -> String

renders type as String. The list of identifiers is the list of all free variables in the expression in order reverse to the order of binding.

readType :: String -> Maybe Type

Reads a Type from a String.

mkType :: [Hypo] -> CId -> [Expr] -> Type

creates a type from list of hypothesises, category and list of arguments for the category. The operation mkType [h_1,...,h_n] C [e_1,...,e_m] will create h_1 -> ... -> h_n -> C e_1 ... e_m

mkHypo :: Type -> Hypo

creates hypothesis for non-dependent type i.e. A

mkDepHypo :: CId -> Type -> Hypo

creates hypothesis for dependent type i.e. (x : A)

mkImplHypo :: CId -> Type -> Hypo

creates hypothesis for dependent type with implicit argument i.e. ({x} : A)

unType :: Type -> ([Hypo], CId, [Expr])

categories :: PGF -> [CId]

List of all categories defined in the given grammar. The categories are defined in the abstract syntax with the 'cat' keyword.

categoryContext :: PGF -> CId -> Maybe [Hypo]

startCat :: PGF -> Type

The start category is defined in the grammar with the 'startcat' flag. This is usually the sentence category but it is not necessary. Despite that there is a start category defined you can parse with any category. The start category definition is just for convenience.

Functions

functions :: PGF -> [CId]

List of all functions defined in the abstract syntax

functionsByCat :: PGF -> CId -> [CId]

List of all functions defined for a given category

functionType :: PGF -> CId -> Maybe Type

The type of a given function

missingLins :: PGF -> Language -> [CId]

List of functions that lack linearizations in the given language.

Expressions & Trees

Tree

type Tree = Expr

Tree is the abstract syntax representation of a given sentence in some concrete syntax. Technically Tree is a type synonym of Expr.

Expr

data Expr

An expression in the abstract syntax of the grammar. It could be both parameter of a dependent type or an abstract syntax tree for for some sentence.

Instances

Eq Expr

Ord Expr

Read Expr

Show Expr

showExpr :: [CId] -> Expr -> String

renders expression as String. The list of identifiers is the list of all free variables in the expression in order reverse to the order of binding.

readExpr :: String -> Maybe Expr

parses String as an expression

mkAbs :: BindType -> CId -> Expr -> Expr

unAbs :: Expr -> Maybe (BindType, CId, Expr)

mkApp :: CId -> [Expr] -> Expr

Constructs an expression by applying a function to a list of expressions

unApp :: Expr -> Maybe (CId, [Expr])

Decomposes an expression into application of function

mkStr :: String -> Expr

Constructs an expression from string literal

unStr :: Expr -> Maybe String

Decomposes an expression into string literal

mkInt :: Int -> Expr

Constructs an expression from integer literal

unInt :: Expr -> Maybe Int

Decomposes an expression into integer literal

mkDouble :: Double -> Expr

Constructs an expression from real number literal

unDouble :: Expr -> Maybe Double

Decomposes an expression into real number literal

mkMeta :: Int -> Expr

Constructs an expression which is meta variable

unMeta :: Expr -> Maybe Int

Checks whether an expression is a meta variable

pExpr :: ReadP Expr

Operations

Linearization

linearize :: PGF -> Language -> Tree -> String

Linearizes given expression as string in the language

linearizeAllLang :: PGF -> Tree -> [(Language, String)]

Linearizes given expression as string in all languages available in the grammar.

linearizeAll :: PGF -> Tree -> [String]

The same as linearizeAllLang but does not return the language.

bracketedLinearize :: PGF -> Language -> Tree -> [BracketedString]

Linearizes given expression as a bracketed string in the language

tabularLinearizes :: PGF -> Language -> Expr -> [[(String, String)]]

Creates a table from feature name to linearization. The outher list encodes the variations

groupResults :: [[(Language, String)]] -> [(Language, [String])]

showPrintName :: PGF -> Language -> CId -> String

Show the printname of function or category

data BracketedString

BracketedString represents a sentence that is linearized as usual but we also want to retain the ''brackets'' that mark the beginning and the end of each constituent.

Constructors

Leaf Token

this is the leaf i.e. a single token

Bracket CId !FId !LIndex CId [Expr] [BracketedString]

this is a bracket. The CId is the category of the phrase. The FId is an unique identifier for every phrase in the sentence. For context-free grammars i.e. without discontinuous constituents this identifier is also unique for every bracket. When there are discontinuous phrases then the identifiers are unique for every phrase but not for every bracket since the bracket represents a constituent. The different constituents could still be distinguished by using the constituent index i.e. LIndex. If the grammar is reduplicating then the constituent indices will be the same for all brackets that represents the same constituent.

type FId = Int

type LIndex = Int

type Token = String

showBracketedString :: BracketedString -> String

Renders the bracketed string as string where the brackets are shown as (S ...) where S is the category.

flattenBracketedString :: BracketedString -> [String]

Parsing

parse :: PGF -> Language -> Type -> String -> [Tree]

Tries to parse the given string in the specified language and to produce abstract syntax expression.

parseAllLang :: PGF -> Type -> String -> [(Language, [Tree])]

Tries to parse the given string with all available languages. The returned list contains pairs of language and list of abstract syntax expressions (this is a list, since grammars can be ambiguous). Only those languages for which at least one parsing is possible are listed.

parseAll :: PGF -> Type -> String -> [[Tree]]

The same as parseAllLang but does not return the language.

parse_ :: PGF -> Language -> Type -> Maybe Int -> String -> (ParseOutput, BracketedString)

The same as parse but returns more detailed information

parseWithRecovery :: PGF -> Language -> Type -> [Type] -> Maybe Int -> String -> (ParseOutput, BracketedString)

This is an experimental function. Use it on your own risk

Evaluation

compute :: PGF -> Expr -> Expr

Converts an expression to normal form

paraphrase :: PGF -> Expr -> [Expr]

Type Checking

The type checker in PGF does both type checking and renaming i.e. it verifies that all identifiers are declared and it distinguishes between global function or type indentifiers and variable names. The type checker should always be applied on expressions entered by the user i.e. those produced via functions like readType and readExpr because otherwise unexpected results could appear. All typechecking functions returns updated versions of the input types or expressions because the typechecking could also lead to metavariables instantiations.

checkType :: PGF -> Type -> Either TcError Type

Check whether a given type is consistent with the abstract syntax of the grammar.

checkExpr :: PGF -> Expr -> Type -> Either TcError Expr

Checks an expression against a specified type.

inferExpr :: PGF -> Expr -> Either TcError (Expr, Type)

Tries to infer the type of a given expression. Note that even if the expression is type correct it is not always possible to infer its type in the GF type system. In this case the function returns the CannotInferType error.

data TcError

If an error occurs in the typechecking phase the type checker returns not a plain text error message but a TcError structure which describes the error.

Constructors

UnknownCat CId

Unknown category name was found.

UnknownFun CId

Unknown function name was found.

WrongCatArgs [CId] Type CId Int Int

A category was applied to wrong number of arguments. The first integer is the number of expected arguments and the second the number of given arguments. The [CId] argument is the list of free variables in the type. It should be used for the showType function.

TypeMismatch [CId] Expr Type Type

The expression is not of the expected type. The first type is the expected type, while the second is the inferred. The [CId] argument is the list of free variables in both the expression and the type. It should be used for the showType and showExpr functions.

NotFunType [CId] Expr Type

Something that is not of function type was applied to an argument.

CannotInferType [CId] Expr

It is not possible to infer the type of an expression.

UnresolvedMetaVars [CId] Expr [MetaId]

Some metavariables have to be instantiated in order to complete the typechecking.

UnexpectedImplArg [CId] Expr

Implicit argument was passed where the type doesn't allow it

UnsolvableGoal [CId] MetaId Type

There is a goal that cannot be solved

Instances

Eq TcError

ppTcError :: TcError -> Doc

Renders the type checking error to a document. See PrettyPrint.

Low level parsing API

data ParseState

An abstract data type whose values represent the current state in an incremental parser.

initState :: PGF -> Language -> Type -> ParseState

Creates an initial parsing state for a given language and startup category.

nextState :: ParseState -> ParseInput -> Either ErrorState ParseState

From the current state and the next token nextState computes a new state, where the token is consumed and the current position is shifted by one. If the new token cannot be accepted then an error state is returned.

getCompletions :: ParseState -> String -> Map Token ParseState

If the next token is not known but only its prefix (possible empty prefix) then the getCompletions function can be used to calculate the possible next words and the consequent states. This is used for word completions in the GF interpreter.

recoveryStates :: [Type] -> ErrorState -> (ParseState, Map Token ParseState)

data ParseInput

The input to the parser is a pair of predicates. The first one piToken selects a token from a list of suggestions from the grammar, actually appears at the current position in the input string. The second one piLiteral recognizes whether a literal with forest id FId could be matched at the current position.

Constructors

ParseInput

Fields

piToken :: forall a. Map Token a -> Maybe a

piLiteral :: FId -> Maybe (CId, Tree, [Token])

simpleParseInput :: Token -> ParseInput

This function constructs the simplest possible parser input. It checks the tokens for exact matching and recognizes only String, Int and Float literals. The Int and Float literals match only if the token passed is some number. The String literal always match but the length of the literal could be only one token.

mkParseInput :: PGF -> Language -> (forall a. b -> Map Token a -> Maybe a) -> [(CId, b -> Maybe (Tree, [Token]))] -> b -> ParseInput

data ParseOutput

This data type encodes the different outcomes which you could get from the parser.

Constructors

ParseFailed Int

The integer is the position in number of tokens where the parser failed.

TypeError [(FId, TcError)]

The parsing was successful but none of the trees is type correct. The forest id (FId) points to the bracketed string from the parser where the type checking failed. More than one error is returned if there are many analizes for some phrase but they all are not type correct.

ParseOk [Tree]

If the parsing and the type checking are successful we get a list of abstract syntax trees. The list should be non-empty.

ParseIncomplete

The sentence is not complete. Only partial output is produced

getParseOutput :: ParseState -> Type -> Maybe Int -> (ParseOutput, BracketedString)

This function extracts the list of all completed parse trees that spans the whole input consumed so far. The trees are also limited by the category specified, which is usually the same as the startup category.

Generation

The PGF interpreter allows automatic generation of abstract syntax expressions of a given type. Since the type system of GF allows dependent types, the generation is in general undecidable. In fact, the set of all type signatures in the grammar is equivalent to a Turing-complete language (Prolog).

There are several generation methods which mainly differ in:

• whether the expressions are sequentially or randomly generated?
• are they generated from a template? The template is an expression containing meta variables which the generator will fill in.
• is there a limit of the depth of the expression? The depth can be used to limit the search space, which in some cases is the only way to make the search decidable.

generateAll :: PGF -> Type -> [Expr]

Generates an exhaustive possibly infinite list of abstract syntax expressions.

generateAllDepth :: PGF -> Type -> Maybe Int -> [Expr]

A variant of generateAll which also takes as argument the upper limit of the depth of the generated expression.

generateFrom :: PGF -> Expr -> [Expr]

Generates a list of abstract syntax expressions in a way similar to generateAll but instead of generating all instances of a given type, this function uses a template.

generateFromDepth :: PGF -> Expr -> Maybe Int -> [Expr]

A variant of generateFrom which also takes as argument the upper limit of the depth of the generated subexpressions.

generateRandom :: RandomGen g => g -> PGF -> Type -> [Expr]

Generates an infinite list of random abstract syntax expressions. This is usefull for tree bank generation which after that can be used for grammar testing.

generateRandomDepth :: RandomGen g => g -> PGF -> Type -> Maybe Int -> [Expr]

A variant of generateRandom which also takes as argument the upper limit of the depth of the generated expression.

generateRandomFrom :: RandomGen g => g -> PGF -> Expr -> [Expr]

Random generation based on template

generateRandomFromDepth :: RandomGen g => g -> PGF -> Expr -> Maybe Int -> [Expr]

Random generation based on template with a limitation in the depth.

Morphological Analysis

type Lemma = CId

type Analysis = String

data Morpho

lookupMorpho :: Morpho -> String -> [(Lemma, Analysis)]

buildMorpho :: PGF -> Language -> Morpho

fullFormLexicon :: Morpho -> [(String, [(Lemma, Analysis)])]

morphoMissing :: Morpho -> [String] -> [String]

morphoKnown :: Morpho -> [String] -> [String]

isInMorpho :: Morpho -> String -> Bool

Tokenizing

mkTokenizer :: PGF -> Language -> String -> Maybe [String]

This is the construction function. Given a PGF and a Language, it extract the lexicon for this language and build a tokenization fst from it.

Visualizations

graphvizAbstractTree :: PGF -> (Bool, Bool) -> Tree -> String

Renders abstract syntax tree in Graphviz format

graphvizParseTree :: PGF -> Language -> GraphvizOptions -> Tree -> String

graphvizDependencyTree :: String -> Bool -> Maybe Labels -> Maybe String -> PGF -> CId -> Tree -> String

graphvizBracketedString :: GraphvizOptions -> [BracketedString] -> String

graphvizAlignment :: PGF -> [Language] -> Expr -> String

gizaAlignment :: PGF -> (Language, Language) -> Expr -> (String, String, String)

data GraphvizOptions

Constructors

GraphvizOptions

Fields

noLeaves :: Bool

noFun :: Bool

noCat :: Bool

nodeFont :: String

leafFont :: String

nodeColor :: String

leafColor :: String

nodeEdgeStyle :: String

leafEdgeStyle :: String

graphvizDefaults :: GraphvizOptions

getDepLabels :: [String] -> Labels

Probabilities

data Probabilities

An abstract data structure which represents the probabilities for the different functions in a grammar.

mkProbabilities :: PGF -> Map CId Double -> Probabilities

Builds probability tables. The second argument is a map which contains the know probabilities. If some function is not in the map then it gets assigned some probability based on the even distribution of the unallocated probability mass for the result category.

defaultProbabilities :: PGF -> Probabilities

Returns the default even distibution.

showProbabilities :: Probabilities -> String

Renders the probability structure as string

readProbabilitiesFromFile :: FilePath -> PGF -> IO Probabilities

Reads the probabilities from a file. This should be a text file where on every line there is a function name followed by a real number. The number represents the probability mass allocated for that function. The function name and the probability should be separated by a whitespace.

probTree :: PGF -> Expr -> Double

compute the probability of a given tree

setProbabilities :: Probabilities -> PGF -> PGF

rankTreesByProbs :: PGF -> [Expr] -> [(Expr, Double)]

rank from highest to lowest probability

Browsing

browse :: PGF -> CId -> Maybe (String, [CId], [CId])

Tries

data ATree

A type for plain applicative trees

Constructors

Other Tree

App CId [ATree]

Instances

Show ATree

data Trie

A type for tries of plain applicative trees

Constructors

Oth Tree

Ap CId [[Trie]]

Instances

Show Trie

toATree :: Tree -> ATree

Convert a Tree to an ATree

toTrie :: [ATree] -> [[Trie]]

Combine a list of trees into a trie