outsort: External sorting package based on Conduit

[ algorithms, mit, program ] [ Propose Tags ]

External (disk-backed) sorting package based on Conduit, saving intermediate files to disk and later merging them all.

[Skip to Readme]
Versions [faq] 0.1.0
Change log ChangeLog
Dependencies async, base (>=4.7 && <5), bytestring, conduit, conduit-algorithms, conduit-combinators, conduit-extra, containers, deepseq, directory, exceptions, filemanip, filepath, MissingH, primitive, resourcet, safe, safeio, temporary, text, transformers, transformers-base, vector, vector-algorithms [details]
License MIT
Maintainer Luis Pedro Coelho <luis@luispedro.org>
Category Algorithms
Uploaded by luispedro at 2019-07-11T06:39:18Z
Distributions NixOS:0.1.0
Executables SortLines
Downloads 382 total (2 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Hackage Matrix CI
Docs not available [build log]
All reported builds failed as of 2019-07-11 [all 2 reports]


Maintainer's Corner

For package maintainers and hackage trustees

Readme for outsort-0.1.0

[back to package description]

Outsort: generic (Haskell-based) external sorting


    import qualified Data.Conduit.Combinators as CC
    import qualified Data.Conduit.Binary as CB

    import Algorithms.OutSort (isolateBySize)
    import Algorithms.SortMain (sortMain)

    main :: IO ()
    main = sortMain
        (isolateBySize (const 1) 500000)

All that is needed is a decoder (ConduitT ByteString a m ()), an encoder (ConduitT ByteString a m ()), and a function to split the input into blocks (ConduitT a a m ()). Given these elements, the result is a programme which can sort arbitrarily large inputs using external memory.

Licence: MIT

Author: Luis Pedro Coelho (email: coelho@embl.de) (on twitter: @luispedrocoelho)