Menoh

Haskell binding for Menoh DNN inference library.

Basic usage

1. Load computation graph from ONNX file using makeModelDataFromONNXFile.
2. Specify input variable type/dimentions (in particular batch size) and which output variables you want to retrieve. This can be done by constructing VariableProfileTable using makeVariableProfileTable.
3. Optimize ModelData with respect to your VariableProfileTable by using optimizeModelData.
4. Construct a Model using makeModel or makeModelWithConfig. If you want to use custom buffers instead of internally allocated ones, You need to use more low level ModelBuilder.
5. Load input data. This can be done conveniently using writeBuffer. There are also more low-level API such as unsafeGetBuffer and withBuffer.
6. Run inference using run.
7. Retrieve the result data. This can be done conveniently using readBuffer.

Note on thread safety

TL;DR: If you want to use Menoh from multiple haskell threads, you need to use threaded RTS by supplying -threaded option to GHC.

Menoh uses thread local storage (TLS) for storing error information, and the only way to use TLS safely is to use in bound threads (see Control.Concurrent).

• In threaded RTS (i.e. rtsSupportsBoundThreads is True), this module runs computation in bound threads by using runInBoundThread. (If the calling thread is not bound, runInBoundThread create a bound thread temporarily and run the computation inside it).
• In non-threaded RTS, this module does not use runInBoundThread and is therefore unsafe to use from multiple haskell threads. Using non-threaded RTS is allowed for the sake of convenience (e.g. running in GHCi) despite its unsafety.

Basic data types

type Dims = [Int]

Dimensions of array

data DType

Data type of array elements

Constructors

DTypeFloat

single precision floating point number

DTypeUnknown !MenohDType

types that this binding is unware of

Instances

Enum DType

Defined in Menoh

Methods

succ :: DType -> DType

pred :: DType -> DType

toEnum :: Int -> DType

fromEnum :: DType -> Int

enumFrom :: DType -> [DType]

enumFromThen :: DType -> DType -> [DType]

enumFromTo :: DType -> DType -> [DType]

enumFromThenTo :: DType -> DType -> DType -> [DType]

Eq DType

Defined in Menoh

Methods

(==) :: DType -> DType -> Bool

(/=) :: DType -> DType -> Bool

Ord DType

Defined in Menoh

Methods

compare :: DType -> DType -> Ordering

(<) :: DType -> DType -> Bool

(<=) :: DType -> DType -> Bool

(>) :: DType -> DType -> Bool

(>=) :: DType -> DType -> Bool

max :: DType -> DType -> DType

min :: DType -> DType -> DType

Read DType

Defined in Menoh

Methods

readsPrec :: Int -> ReadS DType

readList :: ReadS [DType]

readPrec :: ReadPrec DType

readListPrec :: ReadPrec [DType]

Show DType

Defined in Menoh

Methods

showsPrec :: Int -> DType -> ShowS

show :: DType -> String

showList :: [DType] -> ShowS

AddOutput (String, DType)

Defined in Menoh

Methods

addOutput :: VariableProfileTableBuilder -> (String, DType) -> IO ()

data Error

Functions in this module can throw this exception type.

Constructors

ErrorStdError String

ErrorUnknownError String

ErrorInvalidFilename String

ErrorONNXParseError String

ErrorInvalidDType String

ErrorInvalidAttributeType String

ErrorUnsupportedOperatorAttribute String

ErrorDimensionMismatch String

ErrorVariableNotFound String

ErrorIndexOutOfRange String

ErrorJSONParseError String

ErrorInvalidBackendName String

ErrorUnsupportedOperator String

ErrorFailedToConfigureOperator String

ErrorBackendError String

ErrorSameNamedVariableAlreadyExist String

UnsupportedInputDims String

SameNamedParameterAlreadyExist String

SameNamedAttributeAlreadyExist String

InvalidBackendConfigError String

InputNotFoundError String

OutputNotFoundError String

Instances

Eq Error

Defined in Menoh

Methods

(==) :: Error -> Error -> Bool

(/=) :: Error -> Error -> Bool

Ord Error

Defined in Menoh

Methods

compare :: Error -> Error -> Ordering

(<) :: Error -> Error -> Bool

(<=) :: Error -> Error -> Bool

(>) :: Error -> Error -> Bool

(>=) :: Error -> Error -> Bool

max :: Error -> Error -> Error

min :: Error -> Error -> Error

Read Error

Defined in Menoh

Methods

readsPrec :: Int -> ReadS Error

readList :: ReadS [Error]

readPrec :: ReadPrec Error

readListPrec :: ReadPrec [Error]

Show Error

Defined in Menoh

Methods

showsPrec :: Int -> Error -> ShowS

show :: Error -> String

showList :: [Error] -> ShowS

Exception Error

Defined in Menoh

Methods

toException :: Error -> SomeException

fromException :: SomeException -> Maybe Error

displayException :: Error -> String

ModelData type

newtype ModelData

ModelData contains model parameters and computation graph structure.

Constructors

ModelData (ForeignPtr MenohModelData)

makeModelDataFromONNXFile :: MonadIO m => FilePath -> m ModelData

Load onnx file and make ModelData.

makeModelDataFromONNX :: MonadIO m => FilePath -> m ModelData

Deprecated: use makeModelDataFromONNXFile instead

Load onnx file and make ModelData.

makeModelDataFromONNXByteString :: MonadIO m => ByteString -> m ModelData

make ModelData from on-memory ByteString.

optimizeModelData :: MonadIO m => ModelData -> VariableProfileTable -> m ()

Optimize function for ModelData.

This function modify given ModelData.

Manual model data construction API

makeModelData :: MonadIO m => m ModelData

Make empty ModelData

addParameterFromPtr :: MonadIO m => ModelData -> String -> DType -> Dims -> Ptr a -> m ()

Add a new parameter in ModelData

This API is tentative and will be changed in the future.

Duplication of parameter_name is not allowed and it throws error.

addNewNode :: MonadIO m => ModelData -> String -> m ()

Add a new node to ModelData

addInputNameToCurrentNode :: MonadIO m => ModelData -> String -> m ()

Add a new input name to latest added node in ModelData

addOutputNameToCurrentNode :: MonadIO m => ModelData -> String -> m ()

Add a new output name to latest added node in ModelData

class AttributeType value where

A class of types that can be added to nodes using addAttribute.

Minimal complete definition

basicAddAttribute

Methods

basicAddAttribute :: Ptr MenohModelData -> CString -> value -> IO ()

Instances

AttributeType Float

Defined in Menoh

Methods

basicAddAttribute :: Ptr MenohModelData -> CString -> Float -> IO ()

AttributeType Int

Defined in Menoh

Methods

basicAddAttribute :: Ptr MenohModelData -> CString -> Int -> IO ()

AttributeType [Float]

Defined in Menoh

Methods

basicAddAttribute :: Ptr MenohModelData -> CString -> [Float] -> IO ()

AttributeType [Int]

Defined in Menoh

Methods

basicAddAttribute :: Ptr MenohModelData -> CString -> [Int] -> IO ()

addAttribute :: (AttributeType value, MonadIO m) => ModelData -> String -> value -> m ()

Add a new attribute to latest added node in model_data

VariableProfileTable

newtype VariableProfileTable

VariableProfileTable contains information of dtype and dims of variables.

Users can access to dtype and dims via vptGetDType and vptGetDims.

Constructors

VariableProfileTable (ForeignPtr MenohVariableProfileTable)

makeVariableProfileTable

Arguments

:: (AddOutput a, MonadIO m)

=> [(String, DType, Dims)]

input names with dtypes and dims

-> [a]

required output informations (String or (String, DType))

-> ModelData

model data

-> m VariableProfileTable

Convenient function for constructing VariableProfileTable.

If you need finer control, you can use VariableProfileTableBuidler.

vptGetDType :: MonadIO m => VariableProfileTable -> String -> m DType

Accessor function for VariableProfileTable

Select variable name and get its DType.

vptGetDims :: MonadIO m => VariableProfileTable -> String -> m Dims

Accessor function for VariableProfileTable

Select variable name and get its Dims.

Model type

newtype Model

ONNX model with input/output buffers

Constructors

Model (ForeignPtr MenohModel)

makeModel

Arguments

:: MonadIO m

=> VariableProfileTable

variable profile table

-> ModelData

model data

-> String

backend name

-> m Model

Convenient methods for constructing a Model.

makeModelWithConfig

Arguments

:: (MonadIO m, ToJSON a)

=> VariableProfileTable

variable profile table

-> ModelData

model data

-> String

backend name

-> a

backend config

-> m Model

Similar to makeModel but backend-specific configuration can be supplied.

run :: MonadIO m => Model -> m ()

Run model inference.

This function can't be called asynchronously.

getDType :: MonadIO m => Model -> String -> m DType

Get DType of target variable.

getDims :: MonadIO m => Model -> String -> m Dims

Get Dims of target variable.

Accessors for buffers

class ToBuffer a where

Type that can be written to menoh's buffer.

Minimal complete definition

basicWriteBuffer

Methods

basicWriteBuffer :: DType -> Dims -> Ptr () -> a -> IO ()

Instances

ToBuffer a => ToBuffer [a]

Defined in Menoh

Methods

basicWriteBuffer :: DType -> Dims -> Ptr () -> [a] -> IO ()

ToBuffer (Vector Float)

Defined in Menoh

Methods

basicWriteBuffer :: DType -> Dims -> Ptr () -> Vector Float -> IO ()

ToBuffer (Vector Float)

Defined in Menoh

Methods

basicWriteBuffer :: DType -> Dims -> Ptr () -> Vector Float -> IO ()

ToBuffer (Vector Float)

Defined in Menoh

Methods

basicWriteBuffer :: DType -> Dims -> Ptr () -> Vector Float -> IO ()

class FromBuffer a where

Type that can be read from menoh's buffer.

Minimal complete definition

basicReadBuffer

Methods

basicReadBuffer :: DType -> Dims -> Ptr () -> IO a

Instances

FromBuffer a => FromBuffer [a]

Defined in Menoh

Methods

basicReadBuffer :: DType -> Dims -> Ptr () -> IO [a]

FromBuffer (Vector Float)

Defined in Menoh

Methods

basicReadBuffer :: DType -> Dims -> Ptr () -> IO (Vector Float)

FromBuffer (Vector Float)

Defined in Menoh

Methods

basicReadBuffer :: DType -> Dims -> Ptr () -> IO (Vector Float)

FromBuffer (Vector Float)

Defined in Menoh

Methods

basicReadBuffer :: DType -> Dims -> Ptr () -> IO (Vector Float)

writeBuffer :: (ToBuffer a, MonadIO m) => Model -> String -> a -> m ()

Write values to the given model's buffer

readBuffer :: (FromBuffer a, MonadIO m) => Model -> String -> m a

Read values from the given model's buffer

Low-level accessors for buffers

unsafeGetBuffer :: MonadIO m => Model -> String -> m (Ptr a)

Get a buffer handle attached to target variable.

Users can get a buffer handle attached to target variable. If that buffer is allocated by users and attached to the variable by calling attachExternalBuffer, returned buffer handle is same to it.

This function is unsafe because it does not prevent the model to be GC'ed and the returned pointer become dangling pointer.

See also withBuffer.

withBuffer :: forall m r a. (MonadIO m, MonadBaseControl IO m) => Model -> String -> (Ptr a -> m r) -> m r

This function takes a function which is applied to the buffer associated to specified variable. The resulting action is then executed. The buffer is kept alive at least during the whole action, even if it is not used directly inside. Note that it is not safe to return the pointer from the action and use it after the action completes.

See also unsafeGetBuffer.

Deprecated accessors for buffers

class Storable a => HasDType a where

Deprecated: use FromBuffer/ToBuffer instead

Haskell types that have associated DType type code.

Minimal complete definition

dtypeOf

Methods

dtypeOf :: Proxy a -> DType

Instances

HasDType Float

Defined in Menoh

Methods

dtypeOf :: Proxy Float -> DType

HasDType CFloat

Defined in Menoh

Methods

dtypeOf :: Proxy CFloat -> DType

writeBufferFromVector :: forall v a m. (Vector v a, HasDType a, MonadIO m) => Model -> String -> v a -> m ()

Deprecated: Use ToBuffer class and writeBuffer instead

Copy whole elements of Vector into a model's buffer

writeBufferFromStorableVector :: forall a m. (HasDType a, MonadIO m) => Model -> String -> Vector a -> m ()

Deprecated: Use ToBuffer class and writeBuffer instead

Copy whole elements of Vector a into a model's buffer

readBufferToVector :: forall v a m. (Vector v a, HasDType a, MonadIO m) => Model -> String -> m (v a)

Deprecated: Use FromBuffer class and readBuffer instead

Read whole elements of Array and return as a Vector.

readBufferToStorableVector :: forall a m. (HasDType a, MonadIO m) => Model -> String -> m (Vector a)

Deprecated: Use FromBuffer class and readBuffer instead

Read whole eleemnts of Array and return as a Vector.

Misc

version :: Version

Menoh version which was supplied on compilation time via CPP macro.

bindingVersion :: Version

Version of this Haskell binding. (Not the version of Menoh itself)

Low-level API

Builder for VariableProfileTable

newtype VariableProfileTableBuilder

Builder for creation of VariableProfileTable.

Constructors

VariableProfileTableBuilder (ForeignPtr MenohVariableProfileTableBuilder)

makeVariableProfileTableBuilder :: MonadIO m => m VariableProfileTableBuilder

Factory function for VariableProfileTableBuilder.

addInputProfileDims2

Arguments

:: MonadIO m

=> VariableProfileTableBuilder

-> String

-> DType

-> (Int, Int)

(num, size)

-> m ()

Deprecated: use addInputProfileDims instead

Add 2D input profile.

Input profile contains name, dtype and dims (num, size). This 2D input is conventional batched 1D inputs.

addInputProfileDims4

Arguments

:: MonadIO m

=> VariableProfileTableBuilder

-> String

-> DType

-> (Int, Int, Int, Int)

(num, channel, height, width)

-> m ()

Deprecated: use addInputProfileDims instead

Add 4D input profile

Input profile contains name, dtype and dims (num, channel, height, width). This 4D input is conventional batched image inputs. Image input is 3D (channel, height, width).

addOutputName :: MonadIO m => VariableProfileTableBuilder -> String -> m ()

Add output name

Output profile contains name and dtype. Its Dims and DType are calculated automatically, so that you don't need to specify explicitly.

addOutputProfile :: MonadIO m => VariableProfileTableBuilder -> String -> DType -> m ()

Deprecated: use addOutputName instead

Add output profile

Output profile contains name and dtype. Its Dims are calculated automatically, so that you don't need to specify explicitly.

class AddOutput a where

Type class for abstracting addOutputProfile and addOutputName.

Minimal complete definition

addOutput

Methods

addOutput :: VariableProfileTableBuilder -> a -> IO ()

Instances

AddOutput String

Defined in Menoh

Methods

addOutput :: VariableProfileTableBuilder -> String -> IO ()

AddOutput (String, DType)

Defined in Menoh

Methods

addOutput :: VariableProfileTableBuilder -> (String, DType) -> IO ()

buildVariableProfileTable :: MonadIO m => VariableProfileTableBuilder -> ModelData -> m VariableProfileTable

Factory function for VariableProfileTable

Builder for Model

newtype ModelBuilder

Helper for creating of Model.

Constructors

ModelBuilder (ForeignPtr MenohModelBuilder)

makeModelBuilder :: MonadIO m => VariableProfileTable -> m ModelBuilder

Factory function for ModelBuilder

attachExternalBuffer :: MonadIO m => ModelBuilder -> String -> Ptr a -> m ()

Attach a buffer which allocated by users.

Users can attach a external buffer which they allocated to target variable.

Variables attached no external buffer are attached internal buffers allocated automatically.

Users can get that internal buffer handle by calling unsafeGetBuffer etc. later.

buildModel

Arguments

:: MonadIO m

=> ModelBuilder

-> ModelData

-> String

backend name

-> m Model

Factory function for Model.

buildModelWithConfig

Arguments

:: (MonadIO m, ToJSON a)

=> ModelBuilder

-> ModelData

-> String

backend name

-> a

backend config

-> m Model

Similar to buildModel, but backend specific configuration can be supplied as JSON.