flat: Principled and efficient bit-oriented binary serialization.

Haskell implementation of Flat, a principled, portable and compact binary data format (specs).

How To Use It For Fun and Profit

{-# LANGUAGE DeriveGeneric, DeriveAnyClass #-}

import Flat

data Direction = North | South | Center | East | West deriving (Show,Generic,Flat)

Use flat to encode:

flat $ [North,South]
-> "\149"

and unflat to decode:

unflat (flat $ [North,South]) :: Decoded [Direction]
-> Right [ North , South ]

And thanks to Flat's bit-encoding, this little list fits in a single byte (rather than the five that would be required by a traditional byte encoding):

flatBits $ [North,South]
-> "10010101"

Performance

For some hard data, see this comparison of the major haskell serialisation libraries.

Briefly:

• Size: flat produces significantly smaller binaries than all other libraries (3/4 times usually)
• Serialization time: store, persist and flat are faster
• Deserialization time: store, flat, persist and cereal are faster
• Transfer time (serialisation time + transport time on the network + deserialisation at the receiving end): flat is usually faster for all but the highest network speeds

Documentation

• Tutorial

• Hackage Package and Docs

• Flat Format Specification

Installation

Get the latest stable version from hackage.

Compatibility

Tested with:

• GHC 7.10.3 to 9.4.2 (x64)

• GHCJS

  • Note: versions of flat prior to 0.33 encode Double values incorrectly when they are not aligned with a byte boundary.

Known Bugs and Infelicities

• Data types with more than 512 constructors are currently unsupported

• Longish compilation times

  • flat relies more than other serialisation libraries on extensive inlining for its good performance, this unfortunately leads to longer compilation times.

    If you have many data types or very large ones this might become an issue.

    A couple of good practices that will eliminate or mitigate this problem are:

    • During development, turn optimisations off (stack --fast or -O0 in the cabal file).

    • Keep your serialisation code in a separate module or modules.

• See also the full list of open issues.

Ports for other languages

Rust and TypeScript-JavaScript ports are under development.

Get in touch if you would like to help porting flat to other languages.

Acknowledgements

flat reuses ideas and readapts code from various packages, mainly: store, binary-bits and binary and includes bug fixes from a number of contributors.

Other Stuff You Might Like

To decode flat encoded data you need to know the type of the serialised data.

This is ok for applications that do not require long-term storage and that do not operate in open distributed systems.

For those who do, you might want to supplement flat with ZM - Language independent, reproducible, absolute types.