Database.VCache.VRef

Documentation

data VRef a

A VRef is an opaque reference to an immutable value backed by a file, specifically via LMDB. The primary motivation for VRefs is to support memory-cached values, i.e. very large data structures that should not be stored in all at once in RAM.

The API involving VRefs is conceptually pure.

vref  :: (VCacheable a) => VSpace -> a -> VRef a
deref :: VRef a -> a

Under the hood, each VRef has a 64-bit address and a local cache. When dereferenced, the cache is checked or the value is read from the database then cached. Variants of vref and deref control cache behavior.

VCacheable values may themselves contain VRefs and PVars, storing just the address. Very large structured data is readily modeled by using VRefs to load just the pieces you need. However, there is one major constraint:

VRefs may only represent acyclic structures.

If developers want cyclic structure, they need a PVar in the chain. Alternatively, cycles may be modeled indirectly using explicit IDs.

Besides memory caching, VRefs also utilize structure sharing: all VRefs sharing the same serialized representation will share the same address. Structure sharing enables VRefs to be compared for equality without violating conceptual purity. It also simplifies reasoning about idempotence, storage costs, memoization, etc..

Instances

Eq (VRef a)

Show (VRef a)

VCacheable a => VCacheable (VRef a)

Typeable (* -> *) VRef

vref :: VCacheable a => VSpace -> a -> VRef a

Construct a reference with the cache initially active, i.e. such that immediate deref can access the value without reading from the database. The given value will be placed in the cache unless the same vref has already been constructed.

deref :: VRef a -> a

Dereference a VRef, obtaining its value. If the value is not in cache, it will be read into the database then cached. Otherwise, the value is read from cache and the cache is touched to restart any expiration.

Assuming a valid VCacheable instance, this operation should return an equivalent value as was used to construct the VRef.

vref' :: VCacheable a => VSpace -> a -> VRef a

In some cases, developers can reasonably assume they won't need a value in the near future. In these cases, use the vref' constructor to allocate a VRef without caching the content.

deref' :: VRef a -> a

Dereference a VRef. This will read from the cache if the value is available, but will not update the cache. If the value is not cached, it will be read instead from the persistence layer.

This can be useful if you know you'll only dereference a value once for a given task, or if the datatype involved is cheap to parse (e.g. simple bytestrings) such that there isn't a strong need to cache the parse result.

unsafeVRefAddr :: VRef a -> Address

Each VRef has an numeric address in the VSpace. This address is non-deterministic, and essentially independent of the arguments to the vref constructor. This function is unsafe in the sense that it violates the illusion of purity. However, the VRef address will be stable so long as the developer can guarantee it is reachable.

This function may be useful for memoization tables and similar.

The Show instance for VRef will also show the address.

unsafeVRefRefct :: VRef a -> IO Int

This function allows developers to access the reference count for the VRef that is currently recorded in the database. This may be useful for heuristic purposes. However, caveats are needed:

First, due to structure sharing, a VRef may share an address with VRefs of other types having the same serialized form. Reference counts are at the address level.

Second, because the VCache writer operates in a background thread, the reference count returned here may be slightly out of date.

Third, it is possible that VCache will eventually use some other form of garbage collection than reference counting. This function should be considered an unstable element of the API.

vref_space :: VRef a -> VSpace

virtual address space for VRef

data CacheMode

Cache modes are used when deciding, heuristically, whether to clear a value from cache. These modes don't have precise meaning, but there is a general intention: higher numbered modes indicate that VCache should hold onto a value for longer or with greater priority. In the current implementation, CacheMode is used as a pool of hitpoints in a gaming metaphor: if an entry would be removed, but its mode is greater than zero, the mode is reduced instead.

The default for vref and deref is CacheMode1. Use of vrefc or derefc may specify other modes. Cache mode is monotonic: if the same VRef is deref'd with two different modes, the higher mode will be favored.

Note: Regardless of mode, a VRef that is fully GC'd from the Haskell layer will ensure any cached content is also GC'd.

Constructors

CacheMode0

CacheMode1

CacheMode2

CacheMode3

Instances

Eq CacheMode

Ord CacheMode

Show CacheMode

vrefc :: VCacheable a => CacheMode -> VSpace -> a -> VRef a

Construct a VRef with an alternative cache control mode.

derefc :: CacheMode -> VRef a -> a

Dereference a VRef with an alternative cache control mode.

withVRefBytes :: VRef ByteString -> (Ptr Word8 -> Int -> IO a) -> IO a

Specialized, zero-copy access to a `VRef ByteString`. Access to the given ByteString becomes invalid after returning. This operation may also block the writer if it runs much longer than a single writer batch (though, writer batches are frequently large enough that this shouldn't be a problem if you're careful).

unsafeVRefEncoding :: VRef any -> (Ptr Word8 -> Int -> IO a) -> IO a

Zero-copy access to the raw encoding for any VRef. The given data becomes invalid after returning. This is provided for mostly for debugging purposes, i.e. so you can peek under the hood and see how things are encoded or eyeball the encoding.