packed-data for Haskell

Build, traverse and deserialise packed data in Haskell.

What is this for?

When components of a system exchange data, each component has to make sure that they send data in a way the recipient will be able to read. A simple example is an API, sending JSON data for the client to parse. This process of decoding the data takes time, especially for big objects like HTML pages.

Packed data is data serialised into a binary format that is usable as-is, meaning there is no need to parse it to be able to use it. Another perk of such format is that it can be stored in files easily.

packed-data allows using packed data type-safely, without explicit pointer arithmetic.

A portable library

Unlike other implementations of packed-data-support (e.g. Gibbon), packed-data is a library that does not modify the compiler in any way. It relies solely on already existing libraries (like ByteString), Template Haskell and common GHC extensions. This means that, virtually, packed-data can be used with any version of GHC (as of today, it has been tested GHC 9.2-9.12).

Its API is inspired by an example from the Linear Haskell paper (code available here).

Example

import qualified Data.Packed.Reader as R
import Data.Packed.Reader
import Data.Packed

data Tree a = Leaf a | Node (Tree a) (Tree a)

$(mkPacked ''Tree '[])

-- Compute sum of values in the tree
-- Using the generated 'caseTree' function
sumPacked :: PackedReader '[Tree Int] r Int
sumPacked =
    caseTree -- Generated by Template Haskell
        ( R.do -- If Tree is a Leaf
            !n <- reader
            R.return n
        )
        ( R.do -- If Tree is a Node
            !left <- sumPacked
            !right <- sumPacked
            let !res = left + right
            R.return res
        )
-- Alternatively, you can use generated pattern synonyms
sumPacked = PackedReader $ \case
    PackedLeaf l -> reader `with` l
    PackedNode n -> threadedWith n $ R.do
        !left <- sumPacked2
        !right <- sumPacked2
        let !res = left + right
        R.return res


getSum :: Packed '[Tree Int] -> IO Int
getSum = runReader sumPacked

packTree :: Tree Int -> Packed '[Tree Int] 
packTree = pack 

Take a look at the benchmark directory for more examples.

Documentation is available on Hackage

Benchmark

To run benchmarks, run the following command:

stack bench packed-data:bench:tree-bench
# Saves the report as CSV
stack bench packed-data:bench:tree-bench --ba --csv bench.csv
# Saves the report, and runs a specific test
stach bench packed-data:bench:tree-bench --ba '--csv bench.csv sums'