static: Type-safe and interoperable static values and closures

Serialise closures in a type-safe way that interoperates across binaries.

This package is inspired by distributed-static and GHC's static pointers in GHC.StaticPtr, which came out of the same research. However, we make some significantly-different design choices, described below.

GHC made the design choice to focus on guaranteeing that static values could be passed between nodes if they were running the exact same binary, since they are indexed by 64-bit integers automatically-generated by the compiler. distributed-static attempts to support the same source program compiled by different versions of GHC. As part of this effort to preserve stability, one must pass in a table (RemoteTable) whose keys represent the stability, and whose values are resolved potentially differently across compiler versions.

The need for the caller to pass in a RemoteTable is seen as a liability, so two subsequent packages distributed-closure and static-closure take the opposite approach, ripping out the RemoteTable but doubling down on GHC's choice to guarantee compatibility only across different processes running the exact same binary program. Their uses cases are focused around compute clusters and other forms of centralised distributed computing, where this is easy to achieve and not a problem.

Sometimes security is cited as a reason to have this restriction, but this is a bogus argument. "Guarantee compatibility only across same binary" means "same binary => compatibility" whereas the bogus security argument depends on "compatibility => same binary", which is not true - anyone who analyses your binary will know which numbers to spoof, to convince your program via this interface that they are "running the same code". Guaranteeing "same binary" in an adversial setting is in fact extremely hard and cannot be achieved perfectly; in the real-world it can only be approximated, and should be done so via mechanisms designed for it, not via numbers that are slightly hard to brute force at best and trivial to find out at worst.

This package makes the opposite choice, intended for less restricted and more open distributed computing environments such as the internet and decentralised protocols. In these contexts, the requirement of running the exact same binary program is impossible to achieve in practise. Furthermore, we see it as an advantage that code does not need to be exactly the same - for example, one can serialise a closure and its inputs, upgrade your code, then resume running the closure on the same deserialised input arguments but with a bugfixed closure. The necessity to pass in an explicit RemoteTable (here simply called staticTab) is not a liability, but a useful tool to represent high-level compatibility and interoperability. Two nodes with the same keys in their staticTabs, know that they can talk to each other interoperably even if their implementations differ significantly. One node that wishes to talk to different nodes running different minor versions of the same protocol, could instantiate two different staticTabs with the same keys but different implementations, to handle behavioural nuances between the minor versions. In general, it's a useful bit of metadata to keep around in your program code, and can help you perform smooth upgrades of a non-centralised networking protocol more easily.

There are also a few technical differences between this and distributed-static, some of which could be re-adopted there too:

• We use dependent-types and type-level programming to guarantee type safety, rather than Rank1Dynamic. This enables us to store all possible closure types instead of just rank-1 polymorphic functions, including but not limited to: rank-n functions, functions with constraints, those using higher-kinded types, etc.

• Our serialisation typeclasses are designed to interoperate with many different serialisation frameworks. Instances for Data.Binary and Codec.Serialise are provided here for convenience.

• We have additional Template Haskell splices that support creating static values from top-level definitions that must refer to other static values, whether they be recursive or mutually-recursive or neither. This is achieved using an implementation of mfix for the Q monad.

We did not implement the ability to compose static references. The main reason is that, in our view, the purpose of static closures is to represent which top-level tasks to execute, and the inputs to execute it on. This is the interface or contract of this concept. How you run the task is an implementation detail, and as discussed above, this might be different across different machines or as time passes and we upgrade the code. Therefore it makes no sense to serialise a representation of "task A is the composition of closure B and closure C", because it is irrelevant to the interface.

If your interface is actually "run arbitrary user-defined code" (e.g. in a VM or EDSL evaluator) then it would indeed make sense to support composition, but then you should define your own AST, evaluator, and serialiser for this; and pass the ASTs around as regular runtime values, not static values. Supporting arbitrary ASTs like this is outside of the scope of this library.

Further, in Haskell there are many ways of applying values not just (_ :: a -> b) (_ :: a) :: b, e.g. with constraints, with a combination of static and non-static arguments, with type applications, and so on. Only the simple form (_ :: a -> b) (_ :: a) :: b is likely to be interoperable across multiple languages. Supporting AST statics would therefore unnecessarily restrict how we can implement the behaviour of a static closure in our chosen language.

See unit tests for example usage, e.g. UnitTests

Modules

[Last Documentation]

Flags

Manual Flags

Name	Description	Default
dev	Set compile flags for development	Disabled

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

• static-0.1.0.1.tar.gz [browse] (Cabal source package)
• Package description (as included in the package)

Maintainer's Corner

Package maintainers

• infinity0

For package maintainers and hackage trustees

• edit package information

Candidates

• No Candidates