Maintainer	The Vaultaire Team
Stability	Experimental
Safe Haskell	None
Language	Haskell2010

Vaultaire.Types

Contents

Identification of measurements
Time of a measurement
Namespacing and authentication
Metadata about sources
Caching SourceDicts
Operations with the contents store
Streaming reads
Decoded reads
Writes
Hacks
Conversion to and from wire format
Internal
Convenience/clarity

Description

Overview

When communicating with a Vaultaire installation, you need to serialize requests and deserialize responses. The bytes used over the wire are all formed by making the various types involved instances of class WireFormat.

As it happens, these types are also the same ones used in the internals of Vaultaire as it persists measurements and metadata to disk.

Synopsis

Identification of measurements

newtype Address Source

Constructors

Address
Fields unAddress :: Word64

Instances

Bounded Address
Eq Address
Num Address
Ord Address
Read Address
Show Address
IsString Address
Arbitrary Address
Hashable Address
WireFormat Address	There are assumptions made that the encoding of Address is fixed-length (8 bytes). Changing that will break things subtly.

calculateBucketNumber :: Word64 -> Address -> Word64 Source

isAddressExtended :: Address -> Bool Source

Time of a measurement

newtype TimeStamp Source

Number of nanoseconds since the Unix epoch, stored in a Word64.

The Show instance displays the TimeStamp as seconds with the nanosecond precision expressed as a decimal amount after the interger, ie:

>>> t <- getCurrentTimeNanoseconds
>>> show t
2014-07-31T23:09:35.274387000Z

However this doesn't change the fact the underlying representation counts nanoseconds since epoch:

>>> show $ unTimeStamp t
1406848175274387000

There is a Read instance that is reasonably accommodating.

>>> read "2014-07-31T13:05:04.942089001Z" ::TimeStamp
2014-07-31T13:05:04.942089001Z

>>> read "1406811904.942089001" :: TimeStamp
2014-07-31T13:05:04.942089001Z

>>> read "1406811904" :: TimeStamp
2014-07-31T13:05:04.000000000Z

Constructors

TimeStamp
Fields unTimeStamp :: Word64

Instances

Bounded TimeStamp
Enum TimeStamp
Eq TimeStamp
Integral TimeStamp
Num TimeStamp
Ord TimeStamp
Read TimeStamp
Real TimeStamp
Show TimeStamp
Arbitrary TimeStamp
WireFormat TimeStamp

convertToDiffTime :: TimeStamp -> NominalDiffTime Source

Utility function to convert nanoseconds since Unix epoch to a NominalDiffTime, allowing you to then use the time manipulation functions in Data.Time.Clock

convertToTimeStamp :: UTCTime -> TimeStamp Source

getCurrentTimeNanoseconds :: IO TimeStamp Source

Get the current system time, expressed as a TimeStamp (which is to say, number of nanoseconds since the Unix epoch).

Namespacing and authentication

newtype Origin Source

Origin is a ByteString representing a data origin. It must be between one and eight bytes.

Constructors

Origin
Fields unOrigin :: ByteString

Instances

Eq Origin
Ord Origin
Read Origin
Show Origin
IsString Origin
Arbitrary Origin
Hashable Origin

makeOrigin :: ByteString -> Either SomeException Origin Source

Metadata about sources

data SourceDict Source

Instances

Eq SourceDict
Show SourceDict
Arbitrary SourceDict
Monoid SourceDict
WireFormat SourceDict

unionSource :: SourceDict -> SourceDict -> SourceDict Source

Wrapped HashMap.union for SourceDicts

diffSource :: SourceDict -> SourceDict -> SourceDict Source

Wrapped HashMap.difference for SourceDicts

lookupSource :: Text -> SourceDict -> Maybe Text Source

Wrapped HashMap.lookup for SourceDicts

makeSourceDict :: HashMap Text Text -> Either String SourceDict Source

Caching SourceDicts

hashSource :: SourceDict -> Word64 Source

Hashes the sourcedict using SipHash Hashes are used primarily to avoid redundant updates

data SourceDictCache Source

Instances

WireFormat SourceDictCache

emptySourceCache :: SourceDictCache Source

insertSourceCache :: Word64 -> SourceDictCache -> SourceDictCache Source

memberSourceCache :: Word64 -> SourceDictCache -> Bool Source

sizeOfSourceCache :: SourceDictCache -> Int Source

Operations with the contents store

data ContentsOperation Source

Constructors

ContentsListRequest
GenerateNewAddress
UpdateSourceTag Address SourceDict
RemoveSourceTag Address SourceDict

Instances

Eq ContentsOperation
Show ContentsOperation
Arbitrary ContentsOperation
WireFormat ContentsOperation

data ContentsResponse Source

Constructors

RandomAddress Address
InvalidContentsOrigin
ContentsListEntry Address SourceDict
EndOfContentsList
UpdateSuccess
RemoveSuccess

Instances

Eq ContentsResponse
Show ContentsResponse
Arbitrary ContentsResponse
WireFormat ContentsResponse

data ContentsListBypass Source

Constructors

ContentsListBypass Address ByteString

Instances

WireFormat ContentsListBypass

Streaming reads

data ReadRequest Source

Constructors

SimpleReadRequest Address TimeStamp TimeStamp
ExtendedReadRequest Address TimeStamp TimeStamp

Instances

Eq ReadRequest
Show ReadRequest
Arbitrary ReadRequest
WireFormat ReadRequest

data ReadStream Source

Constructors

InvalidReadOrigin
SimpleStream
Fields unSimpleStream :: SimpleBurst
ExtendedStream
Fields unExtendedStream :: ExtendedBurst
EndOfStream

Instances

Eq ReadStream
Show ReadStream
Arbitrary ReadStream
WireFormat ReadStream

newtype SimpleBurst Source

Constructors

SimpleBurst
Fields unSimpleBurst :: ByteString

Instances

Eq SimpleBurst
Show SimpleBurst

newtype ExtendedBurst Source

Constructors

ExtendedBurst
Fields unExtendedBurst :: ByteString

Instances

Eq ExtendedBurst
Show ExtendedBurst

Decoded reads

data SimplePoint Source

SimplePoints are simply wrapped packets for Vaultaire Each consists of 24 bytes: An 8 byte Address An 8 byte Timestamp (nanoseconds since Unix epoch) An 8 byte Payload

Constructors

SimplePoint
Fields simpleAddress :: !Address simpleTime :: !TimeStamp simplePayload :: !Word64

Instances

Eq SimplePoint
Show SimplePoint

data ExtendedPoint Source

ExtendedPoints are simply wrapped packets for Vaultaire Each consists of 16 + length bytes: An 8 byte Address An 8 byte Time (in nanoseconds since Unix epoch) A length byte Payload On the wire their equivalent representation takes up 24 + length bytes with format: 8 byte Address, 8 byte Time, 8 byte Length, Payload

Constructors

ExtendedPoint
Fields extendedAddress :: Address extendedTime :: TimeStamp extendedPayload :: ByteString

Instances

Eq ExtendedPoint
Show ExtendedPoint

Writes

data WriteResult Source

Constructors

InvalidWriteOrigin
OnDisk

Instances

Eq WriteResult
Show WriteResult
Arbitrary WriteResult
WireFormat WriteResult

Hacks

newtype PassThrough Source

Constructors

PassThrough
Fields unPassThrough :: ByteString

Instances

Eq PassThrough
Show PassThrough
Arbitrary PassThrough
WireFormat PassThrough

Conversion to and from wire format

class WireFormat operation where Source

This typeclass encapsulates all wire encoding/decoding, with the possibility of a decode failing.

Methods

fromWire :: ByteString -> Either SomeException operation Source

toWire :: operation -> ByteString Source

Instances

WireFormat ()	Default dummy wire format
WireFormat Address	There are assumptions made that the encoding of Address is fixed-length (8 bytes). Changing that will break things subtly.
WireFormat ContentsListBypass
WireFormat DayMap
WireFormat PassThrough
WireFormat ReadStream
WireFormat SourceDict
WireFormat ContentsOperation
WireFormat ContentsResponse
WireFormat SourceDictCache
WireFormat TimeStamp
WireFormat ReadRequest
WireFormat TeleMsg
WireFormat TeleResp
WireFormat AgentID
WireFormat WriteResult

Internal

newtype DayMap Source

Constructors

DayMap
Fields unDayMap :: Map Epoch NumBuckets

Instances

Eq DayMap
Show DayMap
Arbitrary DayMap
Monoid DayMap
WireFormat DayMap

data TeleResp Source

Response for a telemetry request, sent by the profiler to clients.

Constructors

TeleResp
Fields _timestamp :: TimeStamp _aid :: AgentID _msg :: TeleMsg

Instances

Eq TeleResp
Show TeleResp
Arbitrary TeleResp
WireFormat TeleResp

data TeleMsg Source

The actual telemetric data, reported by Vaultaire worker threads to their profiler.

Constructors

TeleMsg
Fields _origin :: Origin _type :: TeleMsgType _payload :: Word64

Instances

Eq TeleMsg
Show TeleMsg
Arbitrary TeleMsg
WireFormat TeleMsg

data TeleMsgType Source

Telemetry types. All counts are absolute and all latencies are in milliseconds.

Constructors

WriterSimplePoints	Total number of simple points written since last message
WriterExtendedPoints	Total number of extended points written since last message
WriterRequest	Total number of write requests received since last message
WriterRequestLatency	Mean latency for one request
WriterCephLatency	Mean Ceph latency for one request
ReaderSimplePoints	Total number of simple points read since last message
ReaderExtendedPoints	Total number of extended points read since last message
ReaderRequest	Total number of read requests received since last message
ReaderRequestLatency	Mean latency for one request
ReaderCephLatency	Mean Ceph latency for one request
ContentsEnumerate	Total number of enumerate requests received since last message
ContentsUpdate	Total number of update requests received since last message
ContentsEnumerateLatency	Mean latency for one enumerate request
ContentsUpdateLatency	Mean latency for one update request
ContentsEnumerateCeph	Mean Ceph latency for one enumerate request
ContentsUpdateCeph	Mean Ceph latency for one update request

Instances

Bounded TeleMsgType
Enum TeleMsgType
Eq TeleMsgType
Ord TeleMsgType
Show TeleMsgType
Arbitrary TeleMsgType

data TeleMsgUOM Source

Constructors

Points
Requests
Milliseconds

Instances

Bounded TeleMsgUOM
Enum TeleMsgUOM
Eq TeleMsgUOM
Ord TeleMsgUOM
Show TeleMsgUOM

msgTypeUOM :: TeleMsgType -> TeleMsgUOM Source

Map a telemetry message type onto its associated UOM.

data AgentID Source

ID string associated with a running daemon, so telemetry messages can be associated with the processes which sent them.

Instances

Eq AgentID
Ord AgentID
Show AgentID
Arbitrary AgentID
Monoid AgentID
WireFormat AgentID

agentIDLength :: Int Source

Agent IDs are a maximum of 64 bytes.

agentID :: String -> Maybe AgentID Source

An agent ID has to fit in 64 characters and does not contain NUL.

Convenience/clarity

type Epoch = Word64 Source

type NumBuckets = Word64 Source