protobuf-0.1: Google Protocol Buffers via GHC.Generics

Safe HaskellNone

Data.ProtocolBuffers

Contents

Description

An implementation of Protocol Buffers in pure Haskell.

Extensive documentation is available at https://developers.google.com/protocol-buffers/docs/overview and Google's reference implementation can be found at http://code.google.com/p/protobuf/.

It is intended to be used via GHC.Generics and does not require .proto files to function. Tools are being developed that will convert a Haskell Protobuf definition into a .proto and vise versa.

The Data.TypeLevel dependency is required due to http://hackage.haskell.org/trac/ghc/ticket/7459. I believe the partial fix already committed will allow migrating to GHC.TypeLits once GHC 7.8.1 is released.

Given a message definition: 

{-# LANGUAGE DeriveGeneric #-}

import Data.Int
import Data.ProtocolBuffers
import Data.TypeLevel (D1, D2, D3)
import Data.Text
import GHC.Generics (Generic)

data Foo = Foo
   { field1 :: Required D1 (Value Int64) -- ^ The last field with tag = 1
   , field2 :: Optional D2 (Value Text) -- ^ The last field with tag = 2
   , field3 :: Repeated D3 (Value Bool) -- ^ All fields with tag = 3, ordering is preserved
   } deriving (Generic, Show)

instance Encode Foo
instance Decode Foo

It can then be used for encoding and decoding.

To construct a message, use putField to set each field value. Optional, Repeated and Packed fields can be set to their empty value by using mempty. An example using record syntax for clarity: 

>>> let msg = Foo{field1 = putField 42, field2 = mempty, field3 = putField [True, False]}

To serialize a message first convert it into a Put by way of encodeMessage and then to a ByteString by using runPut. Lazy ByteString serialization is done with runPutLazy. 

>>> fmap hex runPut $ encodeMessage msg
"082A18011800"

Decoding is done with the inverse functions: decodeMessage and runGet, or runGetLazy. 

>>> runGet decodeMessage =<< unhex "082A18011800" :: Either String Foo
Right
  (Foo
    { field1 = Field {runField = Required {runRequired = Always {runAlways = Value {runValue = 42}}}}
    , field2 = Field {runField = Optional {runOptional = Last {getLast = Nothing}}}
    , field3 = Field {runField = Repeated {runRepeated = [Value {runValue = True},Value {runValue = False}]}}
    }
  )

Use getField to read fields from a message: 

>>> let Right msg = runGet decodeMessage =<< unhex "082A18011800" :: Either String Foo
>>> getField $ field1 msg
42
>>> getField $ field2 msg
Nothing
>>> getField $ field3 msg
[True,False]

Some Protocol Buffers features are not currently implemented:

  • Default values for Optional fields
  • Extension fields
  • Storing unknown fields, those without a mapped field tag in message record
  • Tag-delimited Groups, deprecated in lieu of Message

Synopsis

Message Serialization

Encoding

class Encode a whereSource

Methods

encode :: a -> PutSource

Instances

Encode (HashMap Tag [WireField])

Untyped message encoding

encodeMessage :: Encode a => a -> PutSource

Encode a Protocol Buffers message.

encodeLengthPrefixedMessage :: Encode a => a -> PutSource

Encode a Protocol Buffers message prefixed with a zz-encoded 32-bit integer describing it's length.

Decoding

class Decode a whereSource

Methods

decode :: HashMap Tag [WireField] -> Get aSource

Instances

Decode (HashMap Tag [WireField])

Untyped message decoding, decode = id

decodeMessage :: Decode a => Get aSource

Decode a Protocol Buffers message.

decodeLengthPrefixedMessage :: Decode a => Get aSource

Decode a Protocol Buffers message prefixed with a zz-encoded 32-bit integer describing it's length.

Fields

Tags

Restricted type aliases of Field. These are used to attach a field tag (a numeric id) to a field. Each tag must be unique within a given message, though this is not currently checked or enforced.

type family Required n a :: *Source

Required fields. Parsing will return empty if a Required value is not found while decoding.

type family Optional n a :: *Source

Optional fields. Values that are not found will return Nothing.

type Repeated n a = Field n (RepeatedField [a])Source

Lists of values.

type Packed n a = Field n (PackedField (PackedList a))Source

Packed values.

Accessors

Fields tend to have rather complex types that are unpleasant to interact with. HasField was designed to hide this complexity and provide a consistent way of getting and setting fields.

class HasField a whereSource

Functions for wrapping and unwrapping record fields. When applied they will have types similar to these: 

getField :: Required D1 (Value Text) -> Text
putField :: Text -> Required D1 (Value Text)

getField :: Optional D2 (Value Int32) -> Maybe Int32
putField :: Maybe Int32 -> Optional D2 (Value Int32)

getField :: Repeated D3 (Value Double) -> [Double]
putField :: [Double] -> Repeated D3 (Value Double)

getField :: Packed D4 (Value Word64) -> [Word64]
putField :: [Word64] -> Packed D4 (Value Word64)

Associated Types

type FieldType a :: *Source

Methods

getField :: a -> FieldType aSource

Extract a value from it's Field representation.

putField :: FieldType a -> aSource

Wrap it back up again.

field :: Functor f => (FieldType a -> f (FieldType a)) -> a -> f aSource

An isomorphism lens compatible with the lens package

Instances

Selectors

Follow these rules to define fields supported by the generic encoder/decoder:

  • The n phantom type parameter specifies the Protocol Buffers field tag (id).
  • Field tags must be an instance of Nat.
  • Field selectors must be an instance of Foldable to support encoding.
  • Value selectors must be an instance of Monoid to support decoding.

data Field n a Source

Fields are merely a way to hold a field tag along with its type, this shouldn't normally be referenced directly.

This provides better error messages than older versions which used Tagged

Instances

Values

Selectors

Each field value needs to specify the way it should be encoded.

There are three built-in value selectors: Value, Enumeration and Message.

If you're unsure what value selector to use, Value is probably the correct one.

data Value a Source

Value selects the normal/typical way for encoding scalar (primitive) values.

Instances

Functor Value 
Typeable1 Value 
Foldable Value 
Traversable Value 
Bounded a => Bounded (Value a) 
Enum a => Enum (Value a) 
Eq a => Eq (Value a) 
Ord a => Ord (Value a) 
Show a => Show (Value a) 
Monoid a => Monoid (Value a) 
NFData a => NFData (Value a) 
DecodeWire a => DecodeWire (Last (Value a)) 
DecodeWire a => DecodeWire (Maybe (Value a)) 
DecodeWire (PackedList (Value Bool)) 
DecodeWire (PackedList (Value Double)) 
DecodeWire (PackedList (Value Float)) 
DecodeWire (PackedList (Value Int32)) 
DecodeWire (PackedList (Value Int64)) 
DecodeWire (PackedList (Value Word32)) 
DecodeWire (PackedList (Value Word64)) 
DecodeWire (PackedList (Value (Fixed Int32))) 
DecodeWire (PackedList (Value (Fixed Int64))) 
DecodeWire (PackedList (Value (Fixed Word32))) 
DecodeWire (PackedList (Value (Fixed Word64))) 
DecodeWire (PackedList (Value (Signed Int32))) 
DecodeWire (PackedList (Value (Signed Int64))) 
DecodeWire a => DecodeWire (Always (Value a)) 
DecodeWire a => DecodeWire (Value a) 
EncodeWire a => EncodeWire [Value a] 
EncodeWire a => EncodeWire (Last (Value a)) 
EncodeWire a => EncodeWire (Maybe (Value a)) 
EncodeWire (PackedList (Value Bool)) 
EncodeWire (PackedList (Value Double)) 
EncodeWire (PackedList (Value Float)) 
EncodeWire (PackedList (Value Int32)) 
EncodeWire (PackedList (Value Int64)) 
EncodeWire (PackedList (Value Word32)) 
EncodeWire (PackedList (Value Word64)) 
EncodeWire (PackedList (Value (Fixed Int32))) 
EncodeWire (PackedList (Value (Fixed Int64))) 
EncodeWire (PackedList (Value (Fixed Word32))) 
EncodeWire (PackedList (Value (Fixed Word64))) 
EncodeWire (PackedList (Value (Signed Int32))) 
EncodeWire (PackedList (Value (Signed Int64))) 
EncodeWire a => EncodeWire (Always (Value a)) 
EncodeWire a => EncodeWire (Value a) 
HasField (Field n (PackedField (PackedList (Value a))))

Iso: FieldType (Packed n (Value a)) = [a]

HasField (Field n (RepeatedField [Value a]))

Iso: FieldType (Repeated n (Value a)) = [a]

HasField (Field n (OptionalField (Last (Value a))))

Iso: FieldType (Optional n (Value a)) = Maybe a

HasField (Field n (RequiredField (Always (Value a))))

Iso: FieldType (Required n (Value a)) = a

data Enumeration a Source

Enumeration fields use fromEnum and toEnum when encoding and decoding messages.

Instances

data Message m Source

The way to embed a message within another message. These embedded messages are stored as length-delimited fields.

For example: 

data Inner = Inner
   { innerField :: Required D1 (Value Int64)
   } deriving (Generic, Show)

instance Encode Inner
instance Decode Inner

data Outer = Outer
   { outerField :: Required D1 (Message Inner)
   } deriving (Generic, Show)

instance Encode Outer
instance Decode Outer

Instances

Functor Message 
Foldable Message 
Traversable Message 
Eq m => Eq (Message m) 
Ord m => Ord (Message m) 
Show m => Show (Message m) 
(Generic m, GMessageMonoid (Rep m)) => Monoid (Message m) 
(Generic m, GMessageNFData (Rep m)) => NFData (Message m) 
Decode m => DecodeWire (Message m) 
(Foldable f, Encode m) => EncodeWire (f (Message m)) 
HasField (Field n (RepeatedField [Message a]))

Iso: FieldType (Repeated n (Message a)) = [a]

HasField (Field n (OptionalField (Maybe (Message a))))

Iso: FieldType (Optional n (Message a)) = Maybe a

HasField (Field n (RequiredField (Always (Message a))))

Iso: FieldType (Required n (Message a)) = a

Wire Coding

Some primitive values can be more compactly represented. Fields that typically contain negative or very large numbers should use the Signed or Fixed wrappers to select their respective (efficient) formats.

newtype Signed a Source

Signed integers are stored in a zz-encoded form.

Constructors

Signed a 

Instances

newtype Fixed a Source

Fixed integers are stored in little-endian form without additional encoding.

Constructors

Fixed a 

Instances