symbiote: Data serialization, communication, and operation verification implementation

[ bsd3, data, library, testing ] [ Propose Tags ]

Please see the README on GitHub at https://github.com/athanclark/symbiote#readme


[Skip to Readme]
Versions [faq] 0.0.0, 0.0.0.1, 0.0.1, 0.0.1.1, 0.0.2
Change log ChangeLog.md
Dependencies abides, aeson, async, base (>=4.7 && <5), bytestring, cereal, containers, exceptions, extractable-singleton, monad-control-aligned, mtl, QuickCheck, quickcheck-instances, stm, text, wai-transformers (>=0.1.0), websockets-simple (>=0.2.0), websockets-simple-extra, zeromq4-haskell, zeromq4-simple (>=0.0.0.2) [details]
License BSD-3-Clause
Copyright 2019 Athan Clark
Author Athan Clark
Maintainer athan.clark@gmail.com
Category Data, Testing
Home page https://github.com/athanclark/symbiote#readme
Bug tracker https://github.com/athanclark/symbiote/issues
Source repo head: git clone https://github.com/athanclark/symbiote
Uploaded by athanclark at Sun Oct 13 18:54:38 UTC 2019
Distributions NixOS:0.0.2
Downloads 342 total (45 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Hackage Matrix CI
Docs not available [build log]
All reported builds failed as of 2019-10-13 [all 3 reports]

Modules

  • Test
    • Serialization
      • Test.Serialization.Symbiote
        • Test.Serialization.Symbiote.Abides
        • Test.Serialization.Symbiote.Aeson
        • Test.Serialization.Symbiote.Cereal
          • Test.Serialization.Symbiote.Cereal.Lazy
        • Test.Serialization.Symbiote.Core
        • Test.Serialization.Symbiote.Debug
        • Test.Serialization.Symbiote.WebSocket
        • Test.Serialization.Symbiote.ZeroMQ

Downloads

Maintainer's Corner

For package maintainers and hackage trustees


Readme for symbiote-0.0.2

[back to package description]

Symbiote

This project aims to be a network agnostic and data format agnostic serialization verifier - it's main purpose is to verify that data, operations on that data, and serialization of that data is all consistent for multiple platforms. This project defines a trivial protocol that can be re-implemented in any platform, over any network medium.

How the Protocol Works

It's a pretty simple idea:

Setting

Basically, the test suite envolves two peers - we'll call them Peer A and Peer B, and they're connected by some network. Likewise, they are both using (possibly different) systems, which we've denoted as Lang A (like Haskell) and Lang B (like PureScript / JavaScript).

The goal of this suite is to ensure that the same idea defined on these different platforms have the same representation - both operationally (i.e., QuickCheck laws) and through some encoding (like printing the data to JSON or a ByteString).

In the diagram above, each system has some idea of what "Type T" looks and feels like, but could be completely different implementations on each system. Again, though, we care about their outcome being identical; so we disambiguate the idea from the data.

Protocol

The first step in the protocol is for the generating party (Peer A) to generate an instance of Type T, and an operation on that type - this operation, too, needs a serialization implementation (but for most intents and purposes, the implementation will be only for the scope of use with this test suite).

Peer A then does two things: it serializes both the operation and instance to the network-acceptable format, and it "performs" the operation on that instance to get an expected result. Basically, Peer A is using Peer B as a remote procedure call for that operation, and verifying that the result obtained remotely is identical to the result obtained locally.

Remote Procedure Call

As just stated, Peer A then sends the serialized instance and operation to Peer B over that network, where Peer B decodes the data, and performs the operation. Peer B then encodes the result, and sends it back to Peer A.

Verification

Peer A now receives the performed data from Peer B, and can now decode it, and test to see if it is equal to the expected result. If it is, it will tell Peer B that it's "their turn" to generate data.

All of this relies on QuickCheck to generate the data - with the size of the generated data increasing at each step, until a maximum is reached (which is clarified in the documentation).


For an example, check out the tests - they are performed both locally in an asynchronous channel, and remotely with its sister package - purescript-symbiote.