streamly: Beautiful Streaming, Concurrent and Reactive Composition
Streamly is a framework for writing programs in a high level, declarative data flow programming paradigm. It provides a simple API, very close to standard Haskell lists. A program is expressed as a composition of data processing pipes, generally known as streams. Streams can be generated, merged, chained, mapped, zipped, and consumed concurrently – enabling a high level, declarative yet concurrent composition of programs. Programs can be concurrent or non-concurrent without any significant change. Concurrency is auto scaled based on consumption rate. Programmers do not have to be aware of threads, locking or synchronization to write scalable concurrent programs. Streamly provides C like performance, orders of magnitude better compared to existing streaming libraries.
Streamly is designed to express the full spectrum of programs with highest performance. Do not think that if you are writing a small and simple program it may not be for you. It expresses a small "hello world" program with the same efficiency, simplicity and elegance as a large scale concurrent application. It unifies many different aspects of special purpose libraries into a single yet simple framework.
Streamly covers the functionality provided by Haskell lists as well as the
functionality provided by streaming libraries like
conduit with a simpler API and
better performance. Streamly provides
advanced stream composition including various ways of appending, merging,
zipping, splitting, grouping, distributing, partitioning and unzipping of
streams with true streaming and with concurrency. Streamly subsumes the
functionality of list transformer libraries like
list-t and also the logic
programming library logict.
The grouping, splitting and windowing combinators in streamly can be compared
to the window operators in Apache Flink.
However, compared to Flink streamly has a pure functional, succinct and
The concurrency capabilities of streamly are much more advanced and powerful compared to the basic concurrency functionality provided by the async package. Streamly is a first class reactive programming library. If you are familiar with Reactive Extensions you will find that it is very similar. For most RxJs combinators you can find or write corresponding ones in streamly. Streamly can be used as an alternative to Yampa or reflex as well.
Streamly focuses on practical engineering with high performance. From well
written streamly programs one can expect performance competitive to C. High
performance streaming eliminates the need for string and text libraries like
text and their lazy and strict
flavors. The confusion and cognitive overhead arising from different
string types is eliminated. The two fundamental types in streamly are arrays
for storage and streams for processing. Strings and text are simply streams
or arrays of
Char as they should be. Arrays in streamly have performance
at par with the vector library.
Where to find more information:
Quick Overview: README file in the package
Building: Build guide for optimal performance
Detailed Tutorial: Streamly.Tutorial module in the haddock documentation
Interoperation: Streamly.Tutorial module for interop with other streaming libraries
Reference Documentation: Haddock documentation for the respective modules
Examples: examples directory in the package
Guides: docs directory in the package, for documentation on advanced topics, limitations, semantics of the library or on specific use cases.
[Skip to Readme]
|Versions [faq]||0.1.0, 0.1.1, 0.1.2, 0.2.0, 0.2.1, 0.3.0, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.5.2, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2|
|Dependencies||atomic-primops (==0.8.*), base (>=4.8 && <5), containers (>=0.5 && <0.7), deepseq (>=1.4.1 && <1.5), directory (>=1.2.2 && <1.4), exceptions (>=0.8 && <0.11), fusion-plugin-types (==0.1.*), ghc-prim (>=0.2 && <0.7), heaps (==0.3.*), lockfree-queue (>=0.2.3 && <0.3), monad-control (>=1.0 && <2), mtl (>=2.2 && <3), network (>=2.6 && <4), primitive (>=0.5.4 && <0.8), semigroups (==0.18.*), transformers (>=0.4 && <0.6), transformers-base (==0.4.*) [details]|
|Copyright||2017 Harendra Kumar|
|Category||Control, Concurrency, Streaming, Reactivity|
|Source repo||head: git clone https://github.com/composewell/streamly|
|Uploaded||by pranaysashank at 2020-04-23T14:14:50Z|
|Distributions||LTSHaskell:0.5.2, NixOS:0.7.2, Stackage:0.6.1|
|Executables||CamelCase, WordCount, WordClassifier, FromFileClient, FileSinkServer, EchoServer, FileIOExamples, HandleIO, ControlFlow, CirclingSquare, AcidRain, MergeSort, ListDir, SearchQuery|
|Downloads||6592 total (256 in the last 30 days)|
|Rating||2.5 (votes: 5) [estimated by Bayesian average]|
Docs available [build log]
Last success reported on 2020-04-23 [all 1 reports]
Use fusion plugin for benchmarks and executables
Enable inspection testing
Debug build with asserts enabled
Use llvm backend for code generation
Disable rewrite rules for stream fusion
Use CPS style streams when possible
Build including examples
Build including SDL examples
Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info
For package maintainers and hackage trustees