data-dispersal: Space-efficient and privacy-preserving data dispersal algorithms.

[ cryptography, data, lgpl, library ] [ Propose Tags ]

This library provides space-efficient (m,n)-information dispersal algorithms (IDAs).

Given a ByteString bstr of length D, we encode bstr as a list fs of n Fragments, each containing a ByteString of length O(D/m). Then, each fragment in fs could be stored on a separate machine for fault-tolerance. Even if up to n-m of these machines crash, we can still reconstruct the original ByteString out of the remaining m fragments. The total space required for the n fragments is O((n/m)*D). Note that m and n are roughly in the same order, so the actual storage overhead for getting good fault-tolerance increases only by a constant factor.

The module Data.IDA contains the basic information dispersal algorithm. The module Crypto.IDA augments the dispersal scheme by combining it with secret sharing, i.e., the knowledge of up to m-1 fragments does not leak any information about the original data. See Crypto.IDA for details.

GHCi Example:

> :m + Data.IDA
> let msg = Data.ByteString.Char8.pack "my really important data"
> let fragments = encode 5 15 msg
-- Now we could distributed the fragments on different sites to add some
-- fault-tolerance.
> let frags' = drop 5 $ take 10 fragments -- let's pretend that 10 machines crashed
> decode frags'
"my really important data"

Fault-Tolerance:

Suppose that we have N machines and encode our data as 2log(N) fragments with reconstruction threshold m = log(N). Let's assume that we store each fragment on a separate machine and each machine fails (independently) with probability at most 0.5.

This library is based on the following works:

Versions 1.0.0.0, 1.0.0.1, 1.0.0.2
Dependencies AES (>=0.2.9), array (>=0.4.0.1), base (==4.6.*), binary (>=0.7.2.1), bytestring (>=0.10.0.2), entropy (>=0.3.2), finite-field (>=0.8.0), matrix (>=0.3.4.0), secret-sharing (>=1.0.0.0), syb (>=0.4.0), vector (>=0.10.11.0) [details]
License LGPL-2.1-only
Copyright Peter Robinson 2014
Author Peter Robinson <peter.robinson@monoid.at>
Maintainer peter.robinson@monoid.at
Category Data, Cryptography
Home page http://monoid.at/code
Uploaded by PeterRobinson at Mon Sep 8 16:32:19 UTC 2014
Distributions NixOS:1.0.0.2
Downloads 1208 total (14 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2016-12-13 [all 6 reports]
Hackage Matrix CI

Modules

  • Crypto
    • Crypto.IDA
  • Data
    • Data.IDA
      • Data.IDA.FiniteField
      • Data.IDA.Internal

Downloads

Maintainer's Corner

For package maintainers and hackage trustees