Dflow: Processing Real-time event streams

[ bsd3, library, reactivity ] [ Propose Tags ]

This library provides Real Time Stream Processors (RTSPs). An RTSP transforms an input event stream into an output event stream. The output events occur asynchronously with input events. RTSPs can be composed into pipelines or executed in parallel and their outputs merged. A Real Time Action (RTA) monad is provided for creating new primitive RTSPs.

[Skip to Readme]




Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees


  • No Candidates
Versions [RSS] 0.0.1
Dependencies base (>=4 && <5), containers (>=0.4), QuickCheck (>=2.4), stm, time (>=1.1) [details]
License BSD-3-Clause
Copyright © Paul Johnson 2012
Author Paul Johnson
Maintainer <paul@cogito.org.uk>
Category Reactivity
Uploaded by PaulJohnson at 2012-04-29T14:30:20Z
Reverse Dependencies 1 direct, 0 indirect [details]
Downloads 1387 total (2 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs uploaded by user
Build status unknown [no reports yet]

Readme for Dflow-0.0.1

[back to package description]

Real time event processing using data flow declarations.

The following has been tested using ghc 7.0.4 under Fedora 15 64-bit.


$ cabal configure
$ cabal build
$ cabal install


$ cabal configure --enable-test
$ cabal build
$ cabal test

The tests work by building random networks of event processors and
checking them against an equivalent list implementation.  Occasionally
they seem to hit a pathological case that chews up memory and CPU
time.  If memory usage goes over 1GB then kill the test and try again.

Test Coverage

Edit the Dflow.cabal file and add "-fhpc" to the GHC options for the
test (see comment in the file).  Then do "cabal clean" and follow the
instructions for "Tests" above. Then do:

$ hpc markup ArbTest.tix --destdir=coverage

The coverage report is in coverage/hpc_index.html.


The main data types for the application programmer are:

Event: A value that occurs at a certain time.  For instance an 
"Event Char" might represent a key press.

RTSP: The Real Time Stream Processor.  A value of type "RTSP x y"
takes in events of type "x" and emits events of type "y".  RTSPs can
be strung together into pipelines using "." (or ">>>" if you prefer
your data to flow left-to-right). RTSPs are also Monoids, so you can
fork your data through two parallel RTSPs and then merge the results.

RTA: A monad for building stateful RTSPs.  Convert an RTA into an
RTSP using "execRTA" or "accumulateRTA" depending what you want to
do with pending output events when a new input event arrives.

You can test an RTSP in "fast time" (that is, without waiting for
real-time delays) by using "simulateRTSP". However the argument list
of input events must be finite. Then you can execute the RTSP in real
time using "execRTSP" and be confident that the real time behaviour
will match the fast-time behaviour.

To Do

Events are currently sorted into time order using (in effect) an
O(n) insertion sort. Event streams should use something more
sophisticated internally, such as a heap.

What API, if any, should be exposed for event streams? On one hand 
developers should be able to create new kinds of RTSP primitives
using event streams, but on the other hand anything you can do with 
event streams can be done using RTA anyway.

Figure out some way for RTAs and RTSPs to execute non-blocking IO with
timeouts, probably using continuations in some way. Hence develop true
distribution by having RTSPs on different machines interact in a
declarative manner.

Make state persistent, together with a replay mechanism for lost