Menoh

Haskell binding for Menoh DNN inference library.

Basic usage

1. Load computation graph from ONNX file using makeModelDataFromONNX.

1. Specify input variable type/dimentions (in particular batch size) and which output variables you want to retrieve. These information is represented as VariableProfileTable. Simple way to construct VariableProfileTable is to use makeVariableProfileTable.
2. Optimize ModelData with respect to your VariableProfileTable by using optimizeModelData.
3. Construct a Model using makeModel or makeModelWithConfig. If you want to use custom buffers instead of internally allocated ones, You need to use more low level ModelBuilder.
4. Load input data. This can be done conveniently using writeBufferFromVector or writeBufferFromStorableVector. There are also more low-level API such as unsafeGetBuffer and withBuffer.
5. Run inference using run.
6. Retrieve the result data. This can be done conveniently using readBufferToVector or readBufferToStorableVector.

Basic data types

type Dims = [Int]

Dimensions of array

data DType

Data type of array elements

Constructors

DTypeFloat

single precision floating point number

DTypeUnknown !MenohDType

types that this binding is unware of

Instances

Enum DType

Methods

succ :: DType -> DType

pred :: DType -> DType

toEnum :: Int -> DType

fromEnum :: DType -> Int

enumFrom :: DType -> [DType]

enumFromThen :: DType -> DType -> [DType]

enumFromTo :: DType -> DType -> [DType]

enumFromThenTo :: DType -> DType -> DType -> [DType]

Eq DType

Methods

(==) :: DType -> DType -> Bool

(/=) :: DType -> DType -> Bool

Ord DType

Methods

compare :: DType -> DType -> Ordering

(<) :: DType -> DType -> Bool

(<=) :: DType -> DType -> Bool

(>) :: DType -> DType -> Bool

(>=) :: DType -> DType -> Bool

max :: DType -> DType -> DType

min :: DType -> DType -> DType

Read DType

Methods

readsPrec :: Int -> ReadS DType

readList :: ReadS [DType]

readPrec :: ReadPrec DType

readListPrec :: ReadPrec [DType]

Show DType

Methods

showsPrec :: Int -> DType -> ShowS

show :: DType -> String

showList :: [DType] -> ShowS

class Storable a => HasDType a where

Haskell types that have associated DType type code.

Minimal complete definition

dtypeOf

Methods

dtypeOf :: Proxy a -> DType

Instances

HasDType Float

Methods

dtypeOf :: Proxy * Float -> DType

HasDType CFloat

Methods

dtypeOf :: Proxy * CFloat -> DType

data Error

Functions in this module can throw this exception type.

Constructors

ErrorStdError String

ErrorUnknownError String

ErrorInvalidFilename String

ErrorONNXParseError String

ErrorInvalidDType String

ErrorInvalidAttributeType String

ErrorUnsupportedOperatorAttribute String

ErrorDimensionMismatch String

ErrorVariableNotFound String

ErrorIndexOutOfRange String

ErrorJSONParseError String

ErrorInvalidBackendName String

ErrorUnsupportedOperator String

ErrorFailedToConfigureOperator String

ErrorBackendError String

ErrorSameNamedVariableAlreadyExist String

Instances

Eq Error

Methods

(==) :: Error -> Error -> Bool

(/=) :: Error -> Error -> Bool

Ord Error

Methods

compare :: Error -> Error -> Ordering

(<) :: Error -> Error -> Bool

(<=) :: Error -> Error -> Bool

(>) :: Error -> Error -> Bool

(>=) :: Error -> Error -> Bool

max :: Error -> Error -> Error

min :: Error -> Error -> Error

Read Error

Methods

readsPrec :: Int -> ReadS Error

readList :: ReadS [Error]

readPrec :: ReadPrec Error

readListPrec :: ReadPrec [Error]

Show Error

Methods

showsPrec :: Int -> Error -> ShowS

show :: Error -> String

showList :: [Error] -> ShowS

Exception Error

Methods

toException :: Error -> SomeException

fromException :: SomeException -> Maybe Error

displayException :: Error -> String

ModelData type

newtype ModelData

ModelData contains model parameters and computation graph structure.

Constructors

ModelData (ForeignPtr MenohModelData)

makeModelDataFromONNX :: MonadIO m => FilePath -> m ModelData

Load onnx file and make ModelData.

optimizeModelData :: MonadIO m => ModelData -> VariableProfileTable -> m ()

Optimize function for ModelData.

This function modify given ModelData.

Model type

newtype Model

ONNX model with input/output buffers

Constructors

Model (ForeignPtr MenohModel)

makeModel

Arguments

:: MonadIO m

=> VariableProfileTable

variable profile table

-> ModelData

model data

-> String

backend name

-> m Model

Convenient methods for constructing a Model.

makeModelWithConfig

Arguments

:: (MonadIO m, ToJSON a)

=> VariableProfileTable

variable profile table

-> ModelData

model data

-> String

backend name

-> a

backend config

-> m Model

Similar to makeModel but backend-specific configuration can be supplied.

run :: MonadIO m => Model -> m ()

Run model inference.

This function can't be called asynchronously.

getDType :: MonadIO m => Model -> String -> m DType

Get DType of target variable.

getDims :: MonadIO m => Model -> String -> m Dims

Get Dims of target variable.

unsafeGetBuffer :: MonadIO m => Model -> String -> m (Ptr a)

Get a buffer handle attached to target variable.

Users can get a buffer handle attached to target variable. If that buffer is allocated by users and attached to the variable by calling attachExternalBuffer, returned buffer handle is same to it.

This function is unsafe because it does not prevent the model to be GC'ed and the returned pointer become dangling pointer.

See also withBuffer.

withBuffer :: forall m r a. (MonadIO m, MonadBaseControl IO m) => Model -> String -> (Ptr a -> m r) -> m r

This function takes a function which is applied to the buffer associated to specified variable. The resulting action is then executed. The buffer is kept alive at least during the whole action, even if it is not used directly inside. Note that it is not safe to return the pointer from the action and use it after the action completes.

See also unsafeGetBuffer.

writeBufferFromVector :: forall v a m. (Vector v a, HasDType a, MonadIO m) => Model -> String -> v a -> m ()

Copy whole elements of Vector into a model's buffer

writeBufferFromStorableVector :: forall a m. (HasDType a, MonadIO m) => Model -> String -> Vector a -> m ()

Copy whole elements of Vector a into a model's buffer

readBufferToVector :: forall v a m. (Vector v a, HasDType a, MonadIO m) => Model -> String -> m (v a)

Read whole elements of Array and return as a Vector.

readBufferToStorableVector :: forall a m. (HasDType a, MonadIO m) => Model -> String -> m (Vector a)

Read whole eleemnts of Array and return as a Vector Float.

Misc

version :: Version

Menoh version which was supplied on compilation time via CPP macro.

bindingVersion :: Version

Version of this Haskell binding. (Not the version of Menoh itself)

Low-level API

VariableProfileTable

newtype VariableProfileTable

VariableProfileTable contains information of dtype and dims of variables.

Users can access to dtype and dims via vptGetDType and vptGetDims.

Constructors

VariableProfileTable (ForeignPtr MenohVariableProfileTable)

makeVariableProfileTable

Arguments

:: MonadIO m

=> [(String, DType, Dims)]

input names with dtypes and dims

-> [(String, DType)]

required output name list with dtypes

-> ModelData

model data

-> m VariableProfileTable

Convenient function for constructing VariableProfileTable.

If you need finer control, you can use VariableProfileTableBuidler.

vptGetDType :: MonadIO m => VariableProfileTable -> String -> m DType

Accessor function for VariableProfileTable

Select variable name and get its DType.

vptGetDims :: MonadIO m => VariableProfileTable -> String -> m Dims

Accessor function for VariableProfileTable

Select variable name and get its Dims.

Builder for VariableProfileTable

newtype VariableProfileTableBuilder

Builder for creation of VariableProfileTable.

Constructors

VariableProfileTableBuilder (ForeignPtr MenohVariableProfileTableBuilder)

makeVariableProfileTableBuilder :: MonadIO m => m VariableProfileTableBuilder

Factory function for VariableProfileTableBuilder.

addInputProfileDims2

Arguments

:: MonadIO m

=> VariableProfileTableBuilder

-> String

-> DType

-> (Int, Int)

(num, size)

-> m ()

Add 2D input profile.

Input profile contains name, dtype and dims (num, size). This 2D input is conventional batched 1D inputs.

addInputProfileDims4

Arguments

:: MonadIO m

=> VariableProfileTableBuilder

-> String

-> DType

-> (Int, Int, Int, Int)

(num, channel, height, width)

-> m ()

Add 4D input profile

Input profile contains name, dtype and dims (num, channel, height, width). This 4D input is conventional batched image inputs. Image input is 3D (channel, height, width).

addOutputProfile :: MonadIO m => VariableProfileTableBuilder -> String -> DType -> m ()

Add output profile

Output profile contains name and dtype. Its Dims are calculated automatically, so that you don't need to specify explicitly.

buildVariableProfileTable :: MonadIO m => VariableProfileTableBuilder -> ModelData -> m VariableProfileTable

Factory function for VariableProfileTable

Builder for Model

newtype ModelBuilder

Helper for creating of Model.

Constructors

ModelBuilder (ForeignPtr MenohModelBuilder)

makeModelBuilder :: MonadIO m => VariableProfileTable -> m ModelBuilder

Factory function for ModelBuilder

attachExternalBuffer :: MonadIO m => ModelBuilder -> String -> Ptr a -> m ()

Attach a buffer which allocated by users.

Users can attach a external buffer which they allocated to target variable.

Variables attached no external buffer are attached internal buffers allocated automatically.

Users can get that internal buffer handle by calling unsafeGetBuffer etc. later.

buildModel

Arguments

:: MonadIO m

=> ModelBuilder

-> ModelData

-> String

backend name

-> m Model

Factory function for Model.

buildModelWithConfig

Arguments

:: (MonadIO m, ToJSON a)

=> ModelBuilder

-> ModelData

-> String

backend name

-> a

backend config

-> m Model

Similar to buildModel, but backend specific configuration can be supplied as JSON.