SecondTransfer

This library implements enough of the HTTP/2 to build compliant HTTP/2 servers.

Frame encoding and decoding is done with Kazu Yamamoto's http2 package, our goal here is to sort the HTTP/2 frames according to the protocol.

You can find more detailed information about this library at the page https://www.httptwo.com/second-transfer/ (but notice that the site uses the library and doesn't yet talk HTTP/1.1, so use a modern browser).

The library

• Is concurrent, meaning that you can use amazing Haskell lightweight threads to process the requests.
• Obeys HTTP/2 flow control aspects.
• And gives you freedom to (ab)use the HTTP/2 protocol in all the ways envisioned by the standard. In particular you should be able to process streaming requests (long uploads in POST or PUT requests) and to deliver streaming responses. You should even be able to do both simultaneously.

Setting up TLS for HTTP/2 correctly is enough of a shore, so I have bundled here the TLS setup logic. Before you read any further, ATTENTION: enable always the threaded ghc runtime in your final programs if you want TLS to work.

Here is how you create a very basic HTTP/2 webserver:

import SecondTransfer(
    CoherentWorker
    , Footers
    , DataAndConclusion
    , tlsServeWithALPN
    , http2Attendant
    )
import SecondTransfer.Http2(
      makeSessionsContext
    , defaultSessionsConfig
    )

import Data.Conduit


saysHello :: DataAndConclusion
saysHello = do 
    -- The data in each yield will be automatically split across multiple 
    -- data frames if needed, so you can yield a large block of contents here
    -- if you wish. 
    -- If you do multiple yields, no data will be left buffered between them, 
    -- so that you can for example implement a chat client in a single HTTP/2 stream.
    -- Pity browsers hardly support that.
    yield "Hello world!"
    -- No footers
    return []


helloWorldWorker :: CoherentWorker
helloWorldWorker request = return (
    [
        (":status", "200")
    ],
    [], -- No pushed streams
    saysHello
    )


-- For this program to work, it should be run from the top of 
-- the developement directory.
main = do 
    sessions_context <- makeSessionsContext defaultSessionsConfig
    let 
        http2_attendant = http2Attendant sessions_context helloWorldWorker
    tlsServeWithALPN
        "tests/support/servercert.pem"   -- Server certificate
        "tests/support/privkey.pem"      -- Certificate private key
        "127.0.0.1"                      -- On which interface to bind
        [
            ("h2-14", http2_attendant),  -- Protocols present in the ALPN negotiation
            ("h2",    http2_attendant)   -- they may be slightly different, but for this 
                                         -- test it doesn't matter.
        ]
        8000 

CoherentWorker is the type of the basic callback function that you need to implement. The callback is used to handle all requests to the server on a given negotiated ALPN protocol. If you need routing functionality (and you most certainly will need it), you need to build that functionality inside the callback.

The above program uses a test certificate by a fake certificate authority. The certificate is valid for the server name ("authority", in HTTP/2 lingo) www.httpdos.com. So, in order for the above program to run, you probably need to add an alias to your /etc/hosts file. You also need very up-to-date versions of OpenSSL (I'm using OpenSSL 1.0.2) to be compliant with the cipher suites demanded by HTTP/2. The easiest way to test the above program is using a fairly recent version of curl. If everything is allright, you should be able to do:

   $ curl -k --http2 https://www.httpdos.com:8000/
   Hello world!

Types related to coherent workers

A coherent worker is an abstraction that can dance at the tune of HTTP/2. That is, it should be able to take headers request first, and then a source of data coming in the request (for example, POST data). Even before exhausting the source, the coherent worker can post the response headers, and then create its source for the response data. A coherent worker can also present create streams to push to the client.

type Headers = [Header]

List of headers. The first part of each tuple is the header name (be sure to conform to the HTTP/2 convention of using lowercase) and the second part is the headers contents. This list needs to include the special :method, :scheme, :authority and :path pseudo-headers for requests; and :status (with a plain numeric value represented in ascii digits) for responses.

type HeaderName = ByteString

The name part of a header

type HeaderValue = ByteString

The value part of a header

type Header = (HeaderName, HeaderValue)

The complete header

type Request = (Headers, Maybe InputDataStream)

A request is a set of headers and a request body.... which will normally be empty, except for POST and PUT requests. But this library enforces none of that.

type Footers = FinalizationHeaders

Finalization headers

type CoherentWorker = Request -> IO PrincipalStream

Main type of this library. You implement one of these for your server. Basically this is a callback that the library calls as soon as it has all the headers of a request. For GET requests that's the entire request basically, but for POST and PUT requests this is just before the data starts arriving to the server.

type PrincipalStream = (Headers, PushedStreams, DataAndConclusion)

You use this type to answer a request. The Headers are thus response headers and they should contain the :status pseudo-header. The PushedStreams is a list of pushed streams...(I don't thaink that I'm handling those yet)

type PushedStreams = [IO PushedStream]

A list of pushed streams

type PushedStream = (Headers, Headers, DataAndConclusion)

A pushed stream, represented by a list of request headers, a list of response headers, and the usual response body (which may include final footers (not implemented yet)).

type DataAndConclusion = ConduitM () ByteString IO Footers

A source-like conduit with the data returned in the response. The return value of the conduit is a list of footers. For now that list can be anything (even bottom), I'm not handling it just yet.

type InputDataStream = Source IO ByteString

This is a Source conduit (see Haskell Data.Conduit library from Michael Snoyman) that you can use to retrieve the data sent by the client piece-wise.

type FinalizationHeaders = Headers

Finalization headers. If you don't know what they are, chances are that you don't need to worry about them for now. The support in this library for those are at best sketchy.

Basic utilities for HTTP/2 servers

Configuration

Callback types

type Attendant = PushAction -> PullAction -> CloseAction -> IO ()

A function which takes three arguments: the first one says how to send data (on a socket or similar transport), and the second one how to receive data on said socket. The third argument encapsulates the sequence of steps needed for a clean shutdown.

You can implement one of these to let somebody else supply the push, pull and close callbacks. In this library we supply callbacks for TLS sockets, so that you don't need to go through the drudgery of managing those yourself.

Attendants encapsulate all the session book-keeping functionality, which for HTTP/2 is quite complicated. You use the function http2Attendant to create one of these from a CoherentWorker.

Push, Pull and Close actions

You don't need to do anything with these types if you are using http2Attendant and tlsServeWithALPN. But they are useful if you want to implement your own layer.

type PullAction = IO ByteString

Callback type to pull data from a channel. The same as to PushAction applies to exceptions thrown from there.

type PushAction = ByteString -> IO ()

Callback type to push data to a channel. Part of this interface is the abstract exception type IOProblem. Throw an instance of it from here to notify the session that the connection has been broken. There is no way to signal "normal termination", since HTTP/2's normal termination can be observed at a higher level when a GO_AWAY frame is seen.

type CloseAction = IO ()

Callback that the session calls to realease resources associated with the channels. Take into account that your callback should be able to deal with non-clean shutdowns also, for example, if the connection to the remote peer is severed suddenly.

http2Attendant :: SessionsContext -> CoherentWorker -> Attendant

The type of this function is equivalent to:

     http2Attendant :: CoherentWorker -> PushAction -> PullAction -> CloseAction ->  IO ()

Given a CoherentWorker, this function wraps it with flow control, multiplexing, and state maintenance needed to run an HTTP/2 session.

Notice that this function is using HTTP/2 over TLS. We haven't implemented yet a session handling mechanism for HTTP/1.1 .

data IOProblem

Throw exceptions derived from this (e.g, GenericIOProblem below) to have the HTTP/2 session to terminate gracefully.

Instances

Show IOProblem

Exception IOProblem

Typeable * IOProblem

data GenericIOProblem

A concrete case of the above exception. Throw one of this if you don't want to implement your own type. Use IOProblem in catch signatures.

Instances

Show GenericIOProblem

Exception GenericIOProblem

Typeable * GenericIOProblem

High level OpenSSL functions.

Use these functions to create your TLS-compliant HTTP/2 server in a snap.

tlsServeWithALPN

Arguments

:: FilePath

Path to a certificate the server is going to use to identify itself. Bear in mind that multiple domains can be served from the same HTTP/2 TLS socket, so please create the HTTP/2 certificate accordingly.

-> FilePath

Path to the key of your certificate.

-> String

Name of the network interface where you want to start your server

-> [(String, Attendant)]

List of protocol names and the corresponding Attendant to use for each. This way you can serve both HTTP/1.1 over TLS and HTTP/2 in the same socket. When no ALPN negotiation is present during the negotiation, the first protocol in this list is used.

-> Int

Port to open to listen for connections.

-> IO ()

Simple function to open

tlsServeWithALPNAndFinishOnRequest

Arguments

:: FilePath

-> FilePath

Same as for tlsServeWithALPN

-> String

Same as for tlsServeWithALPN

-> [(String, Attendant)]

Same as for tlsServeWithALPN

-> Int

Same as for tlsServeWithALPN

-> MVar FinishRequest

Finish request, write a value here to finish serving

-> IO ()

Interruptible version of tlsServeWithALPN. Use the extra argument to ask the server to finish: you pass an empty MVar and when you want to finish you just populate it.

data TLSLayerGenericProblem

Exception inheriting from IOProblem. This is thrown by the OpenSSL subsystem to signal that the connection was broken or that otherwise there was a problem at the SSL layer.

Constructors

TLSLayerGenericProblem String

Instances

Show TLSLayerGenericProblem

Exception TLSLayerGenericProblem

Typeable * TLSLayerGenericProblem

data FinishRequest

Singleton type. Used in conjunction with an MVar. If the MVar is full, the fuction tlsServeWithALPNAndFinishOnRequest knows that it should finish at its earliest convenience and call the CloseAction for any open sessions.

Constructors

FinishRequest

Logging

The library uses hslogger for its logging. Since logging is expensive, most of the instrumentation needs to be activated at compile time by activating the "debug" flag. And then you need to configure the loggers. The function enableConsoleLogging configures them to output a lot of information to standard output.

enableConsoleLogging :: IO ()

Activates logging to terminal