streamly-lmdb: Stream data to or from LMDB databases using the streamly library.

[ bsd3, database, library, streaming, streamly ] [ Propose Tags ]

Please see the README on GitHub at

[Skip to Readme]


Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


Versions [RSS] 0.0.1,, 0.1.0, 0.2.0, 0.2.1, 0.3.0, 0.4.0, 0.5.0
Change log
Dependencies async (>=2.2.2 && <2.3), base (>=4.7 && <5), bytestring (>= && <0.11), streamly (>=0.8 && <0.9) [details]
License BSD-3-Clause
Copyright 2022 Shlok Datye
Author Shlok Datye
Category Database, Streaming, Streamly
Home page
Bug tracker
Source repo head: git clone
Uploaded by shlok at 2022-07-12T13:36:47Z
Distributions NixOS:0.5.0
Downloads 756 total (11 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs uploaded by user
Build status unknown [no reports yet]

Readme for streamly-lmdb-0.5.0

[back to package description]


Hackage CI

Stream data to or from LMDB databases using the Haskell streamly library.


Install LMDB on your system:

  • Debian Linux: sudo apt-get install liblmdb-dev.
  • macOS: brew install lmdb.

Quick start

{-# LANGUAGE OverloadedStrings #-}

module Main where

import Streamly.External.LMDB
  ( Limits (mapSize),
    WriteOptions (writeTransactionSize),
import qualified Streamly.Prelude as S

main :: IO ()
main = do
  -- Open an environment. There should already exist a file or
  -- directory at the given path. (Empty for a new environment.)
  env <-
    openEnvironment "/path/to/lmdb-database" $
      defaultLimits {mapSize = tebibyte}

  -- Get the main database.
  -- Note: It is common practice with LMDB to create the database
  -- once and reuse it for the remainder of the program’s execution.
  db <- getDatabase env Nothing

  -- Stream key-value pairs into the database.
  let fold' = writeLMDB db defaultWriteOptions {writeTransactionSize = 1}
  let writeStream = S.fromList [("baz", "a"), ("foo", "b"), ("bar", "c")]
  _ <- S.fold fold' writeStream

  -- Stream key-value pairs out of the
  -- database, printing them along the way.
  -- Output:
  --     ("bar","c")
  --     ("baz","a")
  --     ("foo","b")
  let unfold' = readLMDB db Nothing defaultReadOptions
  let readStream = S.unfold unfold' undefined
  S.mapM_ print readStream


See bench/ Summary (with rough figures from our machine):

  • Reading. For reading a fully cached LMDB database, this library (when unsafeReadLMDB is used instead of readLMDB) has roughly a 15 ns/pair overhead compared to plain Haskell IO code, which has roughly another 10 ns/pair overhead compared to C. (The first two being similar fulfills the promise of streamly and stream fusion.) We deduce that if your total workload per pair takes longer than around 25 ns, your bottleneck will not be your usage of this library as opposed to C.
  • Writing. Writing with plain Haskell IO code and with this library is, respectively, around 30% and 50% slower than writing with C. We have not dug further into these differences because this write performance is currently good enough for our purposes.

Linode; Debian 10, Dedicated 32GB: 16 CPU, 640GB Storage, 32GB RAM.