Database.LevelDB.Higher

Higher LevelDB provides a rich monadic API for working with leveldb (http://code.google.com/p/leveldb) databases. It uses the leveldb-haskell bindings to the C++ library. The LevelDBT transformer is a Reader that maintains a database context with the open database as well as default read and write options. It also manages a concept called a KeySpace, which is a bucket scheme that provides a low (storage) overhead named identifier to segregate data. Finally it wraps a ResourceT which is required for use of leveldb-haskell functions.

The other major feature is the scan function and its ScanQuery structure that provides a map / fold abstraction over the Iterator exposed by leveldb-haskell.

Introduction

Operations take place within a MonadLevelDB which is built with the LevelDBT transformer; the most basic type would be LevelDBT IO which is type aliased as LevelDB. The basic operations are the same as the underlying leveldb-haskell versions except that the DB and Options arguments are passed along by the LevelDB Reader, and the keys are automatically qualified with the KeySpaceId.

 {-# LANGUAGE OverloadedStrings #-}
 import Database.LevelDB.Higher

 runCreateLevelDB "/tmp/mydb" "MyKeySpace" $ do
     put "key:1" "this is a value"
     get "key:1"

Just "this is a value"

Basic types

type Key = ByteString

type Value = ByteString

type Item = (Key, Value)

The basic unit of storage is a Key/Value pair.

type KeySpace = ByteString

A KeySpace is similar concept to a "bucket" in other libraries and database systems. The ByteString for KeySpace can be arbitrarily long without performance impact because the system maps the KeySpace name to a 4-byte KeySpaceId internally which is preprended to each Key. KeySpaces are cheap and plentiful and indeed with this library you cannot escape them (you can supply an empty ByteString to use a default KeySpace, but it is still used). One intended use case is to use the full Key of a parent as the KeySpace of its children (instance data in a time-series for example). This lets you scan over a range-based key without passing over any unneeded items.

type KeySpaceId = ByteString

Basic operations

get :: MonadLevelDB m => Key -> m (Maybe Value)

Get a value from the current DB and KeySpace.

put :: MonadLevelDB m => Key -> Value -> m ()

Put a value in the current DB and KeySpace.

delete :: MonadLevelDB m => Key -> m ()

Delete an entry from the current DB and KeySpace.

Batch operations

runBatch :: MonadLevelDB m => WriterT WriteBatch m () -> m ()

Write a batch of operations - use the write and deleteB functions to add operations to the batch list.

putB :: MonadLevelDB m => Key -> Value -> WriterT WriteBatch m ()

Add a Put operation to a WriteBatch -- for use with runBatch.

deleteB :: MonadLevelDB m => Key -> WriterT WriteBatch m ()

Add a Del operation to a WriteBatch -- for use with runBatch.

Scans

scan

Arguments

:: MonadLevelDB m

=> Key

Key at which to start the scan.

-> ScanQuery a b

query functions to execute -- see ScanQuery docs.

-> m b

Scan the keyspace, applying functions and returning results. Look at the documentation for ScanQuery for more information.

This is essentially a fold left that will run until the scanWhile condition is met or the iterator is exhausted. All the results will be copied into memory before the function returns.

data ScanQuery a b

Structure containing functions used within the scan function. You may want to start with one of the builder/helper funcions such as queryItems, which is defined as:

queryItems = queryBegins { scanInit = []
                         , scanMap = id
                         , scanFold = (:)
                         }

Constructors

ScanQuery

Fields

scanInit :: b

starting value for fold/reduce

scanWhile :: Key -> Item -> b -> Bool

scan will continue until this returns false

scanMap :: Item -> a

map or transform an item before it is reduced/accumulated

scanFilter :: Item -> Bool

filter function - return False to leave this Item out of the result

scanFold :: a -> b -> b

accumulator/fold function e.g. (:)

queryItems :: ScanQuery Item [Item]

A basic ScanQuery helper; this query will find all keys that begin the Key argument supplied to scan, and returns them in a list of Item.

Does not require any function overrides.

queryList :: ScanQuery a [a]

a ScanQuery helper with defaults for queryBegins and a list result; requires a map function e.g.:

 scan "encoded-values:" queryList { scanMap = \(_, v) -> decode v }

queryBegins :: ScanQuery a b

A partial ScanQuery helper; this query will find all keys that begin with the Key argument supplied to scan.

Requires an scanInit, a scanMap and a scanFold function.

queryCount :: Num a => ScanQuery a a

a ScanQuery helper to count items beginning with Key argument.

Context modifiers

withKeySpace :: MonadLevelDB m => KeySpace -> m a -> m a

Use a local keyspace for the operation. e.g.:

 runCreateLevelDB "/tmp/mydb" "MyKeySpace" $ do
    put "somekey" "somevalue"
    withKeySpace "Other KeySpace" $ do
        put "somekey" "someother value"
    get "somekey"

 Just "somevalue"

withOptions :: MonadLevelDB m => RWOptions -> m a -> m a

Local Read/Write Options for the action.

withSnapshot :: MonadLevelDB m => m a -> m a

Run a block of get operations based on a single snapshot taken at the beginning of the action. The snapshot will be automatically released when complete.

This means that you can do put operations in the same block, but you will not see those changes inside this computation.

forkLevelDB :: MonadLevelDB m => LevelDB () -> m ThreadId

Fork a LevelDBT IO action and return ThreadId into the current monad. This uses resourceForkIO to handle the reference counting and cleanup resources when the last thread exits.

Monadic Types and Operations

class (Monad m, MonadThrow m, MonadIO m, Applicative m, MonadResource m, MonadBase IO m) => MonadLevelDB m where

MonadLevelDB class used by all the public functions in this module.

Methods

withDBContext :: (DBContext -> DBContext) -> m a -> m a

Override context for an action - only usable internally for functions like withKeySpace and withOptions.

liftLevelDB :: LevelDBT IO a -> m a

Lift a LevelDBT IO action into the current monad.

Instances

(Monad m, MonadLevelDB m) => MonadLevelDB (ListT m)

(Monad m, MonadLevelDB m) => MonadLevelDB (MaybeT m)

(Monad m, MonadLevelDB m) => MonadLevelDB (IdentityT m)

MonadResourceBase m => MonadLevelDB (LevelDBT m)

(Monad m, MonadLevelDB m) => MonadLevelDB (ContT r m)

(Error e, MonadLevelDB m) => MonadLevelDB (ErrorT e m)

(Monad m, MonadLevelDB m) => MonadLevelDB (ReaderT r m)

(Monad m, MonadLevelDB m) => MonadLevelDB (StateT s m)

(Monad m, MonadLevelDB m) => MonadLevelDB (StateT s m)

(Monoid w, MonadLevelDB m) => MonadLevelDB (WriterT w m)

(Monoid w, MonadLevelDB m) => MonadLevelDB (WriterT w m)

(Monoid w, MonadLevelDB m) => MonadLevelDB (RWST r w s m)

(Monoid w, MonadLevelDB m) => MonadLevelDB (RWST r w s m)

data LevelDBT m a

LevelDBT Transformer provides a context for database operations provided in this module.

This transformer has the same constraints as ResourceT as it wraps ResourceT along with a DBContext Reader.

If you aren't building a custom monad stack you can just use the LevelDB alias.

Instances

MonadTrans LevelDBT

MonadTransControl LevelDBT

MonadBase b m => MonadBase b (LevelDBT m)

MonadBaseControl b m => MonadBaseControl b (LevelDBT m)

Monad m => Monad (LevelDBT m)

Functor m => Functor (LevelDBT m)

Applicative m => Applicative (LevelDBT m)

MonadIO m => MonadIO (LevelDBT m)

MonadResourceBase m => MonadResource (LevelDBT m)

MonadThrow m => MonadThrow (LevelDBT m)

MonadResourceBase m => MonadLevelDB (LevelDBT m)

type LevelDB a = LevelDBT IO a

alias for LevelDBT IO - useful if you aren't building a custom stack.

mapLevelDBT :: (m a -> n b) -> LevelDBT m a -> LevelDBT n b

Map/transform the monad below the LevelDBT

runLevelDB

Arguments

:: MonadResourceBase m

=> FilePath

path to DB to open/create

-> Options

database options to use

-> RWOptions

default read/write ops; use withOptions to override

-> KeySpace

Bucket in which Keys will be unique

-> LevelDBT m a

The actions to execute

-> m a

Build a context and execute the actions; uses a ResourceT internally.

tip: you can use the Data.Default (def) method to specify default options e.g.

 runLevelDB "/tmp/mydb" def (def, def{sync = true}) "My Keyspace" $ do

runLevelDB'

Arguments

:: MonadResourceBase m

=> FilePath

path to DB to open/create

-> Options

database options to use

-> RWOptions

default read/write ops; use withOptions to override

-> KeySpace

Bucket in which Keys will be unique

-> LevelDBT m a

The actions to execute

-> ResourceT m a

Same as runLevelDB but doesn't call runResourceT. This gives you the option to manage that yourself

runCreateLevelDB

Arguments

:: MonadResourceBase m

=> FilePath

path to DB to open/create

-> KeySpace

Bucket in which Keys will be unique

-> LevelDBT m a

The actions to execute

-> m a

A helper for runLevelDB using default Options except createIfMissing=True

Re-exports

runResourceT :: MonadBaseControl IO m => ResourceT m a -> m a

data Options

Constructors

Options

Fields

blockRestartInterval :: !Int

blockSize :: !Int

cacheSize :: !Int

comparator :: !(Maybe Comparator)

compression :: !Compression

createIfMissing :: !Bool

errorIfExists :: !Bool

maxOpenFiles :: !Int

paranoidChecks :: !Bool

writeBufferSize :: !Int

filterPolicy :: !(Maybe (Either BloomFilter FilterPolicy))

Instances

Default Options

data ReadOptions

Constructors

ReadOptions

Fields

verifyCheckSums :: !Bool

fillCache :: !Bool

useSnapshot :: !(Maybe Snapshot)

Instances

Eq ReadOptions

Default ReadOptions

data WriteOptions

Constructors

WriteOptions

Fields

sync :: !Bool

Instances

Eq WriteOptions

Show WriteOptions

Default WriteOptions

type RWOptions = (ReadOptions, WriteOptions)

type WriteBatch = [BatchOp]

def :: Default a => a

The default value for this type.

class Monad m => MonadThrow m

Instances

MonadThrow []

MonadThrow IO

MonadThrow Maybe

~ * e SomeException => MonadThrow (Either e)

MonadThrow m => MonadThrow (ListT m)

MonadThrow m => MonadThrow (MaybeT m)

MonadThrow m => MonadThrow (IdentityT m)

MonadThrow m => MonadThrow (ResourceT m)

MonadThrow m => MonadThrow (LevelDBT m)

MonadThrow m => MonadThrow (ContT r m)

(Error e, MonadThrow m) => MonadThrow (ErrorT e m)

MonadThrow m => MonadThrow (ReaderT r m)

MonadThrow m => MonadThrow (StateT s m)

MonadThrow m => MonadThrow (StateT s m)

(MonadThrow m, Monoid w) => MonadThrow (WriterT w m)

(MonadThrow m, Monoid w) => MonadThrow (WriterT w m)

(MonadThrow m, Monoid w) => MonadThrow (RWST r w s m)

(MonadThrow m, Monoid w) => MonadThrow (RWST r w s m)

type MonadResourceBase m = (MonadBaseControl IO m, MonadThrow m, MonadBase IO m, MonadIO m, Applicative m)