http-client-0.3.0.2: An HTTP client engine, intended as a base layer for more user-friendly packages.

Safe HaskellNone

Network.HTTP.Client.Internal

Contents

Description

Note that this is essentially the "kitchen sink" export module, including many functions intended only to be used internally by this package. No API stability is guaranteed for this module. If you see functions here which you believe should be promoted to a stable API, please contact the author.

Synopsis

Low-level response body handling

makeChunkedReaderSource

Arguments

:: Bool

send headers

-> Connection 
-> IO BodyReader 

brConsume :: BodyReader -> IO [ByteString]Source

Strictly consume all remaining chunks of data from the stream.

Since 0.1.0

Raw connection handling

dummyConnectionSource

Arguments

:: [ByteString]

input

-> IO (Connection, IO [ByteString], IO [ByteString])

conn, output, input

For testing

makeConnectionSource

Arguments

:: IO ByteString

read

-> (ByteString -> IO ())

write

-> IO ()

close

-> IO Connection 

Cookies

updateCookieJarSource

Arguments

:: Response a

Response received from server

-> Request

Request which generated the response

-> UTCTime

Value that should be used as "now"

-> CookieJar

Current cookie jar

-> (CookieJar, Response a)

(Updated cookie jar with cookies from the Response, The response stripped of any "Set-Cookie" header)

This applies receiveSetCookie to a given Response

receiveSetCookieSource

Arguments

:: SetCookie

The SetCookie the cookie jar is receiving

-> Request

The request that originated the response that yielded the SetCookie

-> UTCTime

Value that should be used as "now"

-> Bool

Whether or not this request is coming from an "http" source (not javascript or anything like that)

-> CookieJar

Input cookie jar to modify

-> CookieJar

Updated cookie jar

This corresponds to the algorithm described in Section 5.3 "Storage Model" This function consists of calling generateCookie followed by insertCheckedCookie. Use this function if you plan to do both in a row. generateCookie and insertCheckedCookie are only provided for more fine-grained control.

generateCookieSource

Arguments

:: SetCookie

The SetCookie we are encountering

-> Request

The request that originated the response that yielded the SetCookie

-> UTCTime

Value that should be used as "now"

-> Bool

Whether or not this request is coming from an "http" source (not javascript or anything like that)

-> Maybe Cookie

The optional output cookie

Turn a SetCookie into a Cookie, if it is valid

insertCheckedCookieSource

Arguments

:: Cookie

The SetCookie the cookie jar is receiving

-> CookieJar

Input cookie jar to modify

-> Bool

Whether or not this request is coming from an "http" source (not javascript or anything like that)

-> CookieJar

Updated (or not) cookie jar

Insert a cookie created by generateCookie into the cookie jar (or not if it shouldn't be allowed in)

insertCookiesIntoRequestSource

Arguments

:: Request

The request to insert into

-> CookieJar

Current cookie jar

-> UTCTime

Value that should be used as "now"

-> (Request, CookieJar)

(Ouptut request, Updated cookie jar (last-access-time is updated))

This applies the computeCookieString to a given Request

computeCookieStringSource

Arguments

:: Request

Input request

-> CookieJar

Current cookie jar

-> UTCTime

Value that should be used as "now"

-> Bool

Whether or not this request is coming from an "http" source (not javascript or anything like that)

-> (ByteString, CookieJar)

(Contents of a "Cookie" header, Updated cookie jar (last-access-time is updated))

This corresponds to the algorithm described in Section 5.4 "The Cookie Header"

evictExpiredCookiesSource

Arguments

:: CookieJar

Input cookie jar

-> UTCTime

Value that should be used as "now"

-> CookieJar

Filtered cookie jar

This corresponds to the eviction algorithm described in Section 5.3 "Storage Model"

pathMatches :: ByteString -> ByteString -> BoolSource

This corresponds to the subcomponent algorithm entitled "Path-Match" detailed in section 5.1.4

domainMatches :: ByteString -> ByteString -> BoolSource

This corresponds to the subcomponent algorithm entitled "Domain Matching" detailed in section 5.1.3

defaultPath :: Request -> ByteStringSource

This corresponds to the subcomponent algorithm entitled "Paths" detailed in section 5.1.4

Performing requests

withResponse :: Request -> Manager -> (Response BodyReader -> IO a) -> IO aSource

Perform a Request using a connection acquired from the given Manager, and then provide the Response to the given function. This function is fully exception safe, guaranteeing that the response will be closed when the inner function exits. It is defined as:

 withResponse req man f = bracket (responseOpen req man) responseClose f

It is recommended that you use this function in place of explicit calls to responseOpen and responseClose.

You will need to use functions such as brRead to consume the response body.

Since 0.1.0

httpLbs :: Request -> Manager -> IO (Response ByteString)Source

A convenience wrapper around withResponse which reads in the entire response body and immediately closes the connection. Note that this function performs fully strict I/O, and only uses a lazy ByteString in its response for memory efficiency. If you are anticipating a large response body, you are encouraged to use withResponse and brRead instead.

Since 0.1.0

httpRaw :: Request -> Manager -> IO (Response BodyReader)Source

Get a Response without any redirect following.

responseOpen :: Request -> Manager -> IO (Response BodyReader)Source

The most low-level function for initiating an HTTP request.

The first argument to this function gives a full specification on the request: the host to connect to, whether to use SSL, headers, etc. Please see Request for full details. The second argument specifies which Manager should be used.

This function then returns a Response with a BodyReader. The Response contains the status code and headers that were sent back to us, and the BodyReader contains the body of the request. Note that this BodyReader allows you to have fully interleaved IO actions during your HTTP download, making it possible to download very large responses in constant memory.

An important note: the response body returned by this function represents a live HTTP connection. As such, if you do not use the response body, an open socket will be retained indefinitely. You must be certain to call responseClose on this response to free up resources.

This function automatically performs any necessary redirects, as specified by the redirectCount setting.

Since 0.1.0

responseClose :: Response a -> IO ()Source

Close any open resources associated with the given Response. In general, this will either close an active Connection or return it to the Manager to be reused.

Since 0.1.0

applyCheckStatus :: (Status -> ResponseHeaders -> CookieJar -> Maybe SomeException) -> Response BodyReader -> IO (Maybe SomeException)Source

Apply 'Request'\'s checkStatus and return resulting exception if any.

httpRedirectSource

Arguments

:: Int

redirectCount

-> (Request -> IO (Response BodyReader, Maybe Request))

function which performs a request and returns a response, and possibly another request if there's a redirect.

-> Request 
-> IO (Response BodyReader) 

Redirect loop

Parse response headers

Request helper functions

parseUrl :: MonadThrow m => String -> m RequestSource

Convert a URL into a Request.

This defaults some of the values in Request, such as setting method to GET and requestHeaders to [].

Since this function uses MonadThrow, the return monad can be anything that is an instance of MonadThrow, such as IO or Maybe.

Since 0.1.0

setUriRelative :: MonadThrow m => Request -> URI -> m RequestSource

Add a URI to the request. If it is absolute (includes a host name), add it as per setUri; if it is relative, merge it with the existing request.

getUri :: Request -> URISource

Extract a URI from the request.

Since 0.1.0

setUri :: MonadThrow m => Request -> URI -> m RequestSource

Validate a URI, then add it to the request.

browserDecompress :: ByteString -> BoolSource

Decompress a compressed stream unless the content-type is 'application/x-tar'.

alwaysDecompress :: ByteString -> BoolSource

Always decompress a compressed stream.

addProxy :: ByteString -> Int -> Request -> RequestSource

Add a proxy to the Request so that the Request when executed will use the provided proxy.

Since 0.1.0

applyBasicAuth :: ByteString -> ByteString -> Request -> RequestSource

Add a Basic Auth header (with the specified user name and password) to the given Request. Ignore error handling:

  applyBasicAuth "user" "pass" $ fromJust $ parseUrl url

Since 0.1.0

urlEncodedBody :: [(ByteString, ByteString)] -> Request -> RequestSource

Add url-encoded parameters to the Request.

This sets a new requestBody, adds a content-type request header and changes the method to POST.

Since 0.1.0

needsGunzipSource

Arguments

:: Request 
-> [Header]

response headers

-> Bool 

useDefaultTimeout :: Maybe IntSource

Magic value to be placed in a Request to indicate that we should use the timeout value in the Manager.

Since 1.9.3

Low-level response body handling

getRedirectedRequest :: Request -> ResponseHeaders -> CookieJar -> Int -> Maybe RequestSource

If a request is a redirection (status code 3xx) this function will create a new request from the old request, the server headers returned with the redirection, and the redirection code itself. This function returns Nothing if the code is not a 3xx, there is no location header included, or if the redirected response couldn't be parsed with parseUrl.

If a user of this library wants to know the url chain that results from a specific request, that user has to re-implement the redirect-following logic themselves. An example of that might look like this:

 myHttp req man = do
    (res, redirectRequests) <- (`runStateT` []) $
         'httpRedirect'
             9000
             (\req' -> do
                res <- http req'{redirectCount=0} man
                modify (\rqs -> req' : rqs)
                return (res, getRedirectedRequest req' (responseHeaders res) (responseCookieJar res) (W.statusCode (responseStatus res))
                )
             'lift'
             req
    applyCheckStatus (checkStatus req) res
    return redirectRequests

lbsResponse :: Response BodyReader -> IO (Response ByteString)Source

Convert a Response that has a Source body to one with a lazy ByteString body.

Manager

data ManagerSettings Source

Settings for a Manager. Please use the defaultManagerSettings function and then modify individual settings. For more information, see http://www.yesodweb.com/book/settings-types.

Since 0.1.0

Constructors

ManagerSettings 

Fields

managerConnCount :: !Int

Number of connections to a single host to keep alive. Default: 10.

Since 0.1.0

managerRawConnection :: !(IO (Maybe HostAddress -> String -> Int -> IO Connection))

Create an insecure connection.

Since 0.1.0 FIXME in the future, combine managerTlsConnection and managerTlsProxyConnection

managerTlsConnection :: !(IO (Maybe HostAddress -> String -> Int -> IO Connection))

Create a TLS connection. Default behavior: throw an exception that TLS is not supported.

Since 0.1.0

managerTlsProxyConnection :: !(IO (ByteString -> (Connection -> IO ()) -> Maybe HostAddress -> String -> Int -> IO Connection))

Create a TLS proxy connection. Default behavior: throw an exception that TLS is not supported.

Since 0.2.2

managerResponseTimeout :: !(Maybe Int)

Default timeout (in microseconds) to be applied to requests which do not provide a timeout value.

Default is 5 seconds

Since 0.1.0

managerRetryableException :: !(SomeException -> Bool)

Exceptions for which we should retry our request if we were reusing an already open connection. In the case of IOExceptions, for example, we assume that the connection was closed on the server and therefore open a new one.

Since 0.1.0

managerWrapIOException :: !(forall a. IO a -> IO a)

Action wrapped around all attempted Requests, usually used to wrap up exceptions in library-specific types.

Default: wrap all IOExceptions in the InternalIOException constructor.

Since 0.1.0

newManager :: ManagerSettings -> IO ManagerSource

Create a Manager. You may manually call closeManager to shut it down, or allow the Manager to be shut down automatically based on garbage collection.

Creating a new Manager is a relatively expensive operation, you are advised to share a single Manager between requests instead.

The first argument to this function is often defaultManagerSettings, though add-on libraries may provide a recommended replacement.

Since 0.1.0

closeManager :: Manager -> IO ()Source

Close all connections in a Manager.

Note that this doesn't affect currently in-flight connections, meaning you can safely use it without hurting any queries you may have concurrently running.

Since 0.1.0

withManager :: ManagerSettings -> (Manager -> IO a) -> IO aSource

Create, use and close a Manager.

Since 0.2.1

failedConnectionException :: Request -> HttpExceptionSource

Create an exception to be thrown if the connection for the given request fails.

defaultManagerSettings :: ManagerSettingsSource

Default value for ManagerSettings.

Since 0.1.0

All types

data BodyReader Source

An abstraction for representing an incoming response body coming from the server. Data provided by this abstraction has already been gunzipped and de-chunked, and respects any content-length headers present.

Since 0.1.0

Constructors

BodyReader 

Fields

brRead :: !(IO ByteString)

Get a single chunk of data from the response body, or an empty bytestring if no more data is available.

Since 0.1.0

brComplete :: !(IO Bool)
 

data Connection Source

Constructors

Connection 

Fields

connectionRead :: !(IO ByteString)

If no more data, return empty.

connectionUnread :: !(ByteString -> IO ())

Return data to be read next time.

connectionWrite :: !(ByteString -> IO ())

Send data to server

connectionClose :: !(IO ())
 

newtype CookieJar Source

Constructors

CJ 

Fields

expose :: [Cookie]
 

data Proxy Source

Define a HTTP proxy, consisting of a hostname and port number.

Constructors

Proxy 

Fields

proxyHost :: !ByteString

The host name of the HTTP proxy.

proxyPort :: !Int

The port number of the HTTP proxy.

data RequestBody Source

When using one of the RequestBodyStream / RequestBodyStreamChunked constructors, you must ensure that the GivesPopper can be called multiple times. Usually this is not a problem.

The RequestBodyStreamChunked will send a chunked request body. Note that not all servers support this. Only use RequestBodyStreamChunked if you know the server you're sending to supports chunked request bodies.

Since 0.1.0

Instances

type Popper = IO ByteStringSource

A function which generates successive chunks of a request body, provider a single empty bytestring when no more data is available.

Since 0.1.0

type NeedsPopper a = Popper -> IO aSource

A function which must be provided with a Popper.

Since 0.1.0

type GivesPopper a = NeedsPopper a -> IO aSource

A function which will provide a Popper to a NeedsPopper. This seemingly convoluted structure allows for creation of request bodies which allocate scarce resources in an exception safe manner.

Since 0.1.0

data Request Source

All information on how to connect to a host and what should be sent in the HTTP request.

If you simply wish to download from a URL, see parseUrl.

The constructor for this data type is not exposed. Instead, you should use either the def method to retrieve a default instance, or parseUrl to construct from a URL, and then use the records below to make modifications. This approach allows http-client to add configuration options without breaking backwards compatibility.

For example, to construct a POST request, you could do something like:

 initReq <- parseUrl "http://www.example.com/path"
 let req = initReq
             { method = "POST"
             }

For more information, please see http://www.yesodweb.com/book/settings-types.

Since 0.1.0

Constructors

Request 

Fields

method :: Method

HTTP request method, eg GET, POST.

Since 0.1.0

secure :: Bool

Whether to use HTTPS (ie, SSL).

Since 0.1.0

host :: ByteString

Requested host name, used for both the IP address to connect to and the host request header.

Since 0.1.0

port :: Int

The port to connect to. Also used for generating the host request header.

Since 0.1.0

path :: ByteString

Everything from the host to the query string.

Since 0.1.0

queryString :: ByteString

Query string appended to the path.

Since 0.1.0

requestHeaders :: RequestHeaders

Custom HTTP request headers

The Content-Length and Transfer-Encoding headers are set automatically by this module, and shall not be added to requestHeaders.

If not provided by the user, Host will automatically be set based on the host and port fields.

Moreover, the Accept-Encoding header is set implicitly to gzip for convenience by default. This behaviour can be overridden if needed, by setting the header explicitly to a different value. In order to omit the Accept-Header altogether, set it to the empty string "". If you need an empty Accept-Header (i.e. requesting the identity encoding), set it to a non-empty white-space string, e.g. " ". See RFC 2616 section 14.3 for details about the semantics of the Accept-Header field. If you request a content-encoding not supported by this module, you will have to decode it yourself (see also the decompress field).

Note: Multiple header fields with the same field-name will result in multiple header fields being sent and therefore it's the responsibility of the client code to ensure that the rules from RFC 2616 section 4.2 are honoured.

Since 0.1.0

requestBody :: RequestBody

Request body to be sent to the server.

Since 0.1.0

proxy :: Maybe Proxy

Optional HTTP proxy.

Since 0.1.0

hostAddress :: Maybe HostAddress

Optional resolved host address. May not be used by all backends.

Since 0.1.0

rawBody :: Bool

If True, a chunked and/or gzipped body will not be decoded. Use with caution.

Since 0.1.0

decompress :: ByteString -> Bool

Predicate to specify whether gzipped data should be decompressed on the fly (see alwaysDecompress and browserDecompress). Argument is the mime type. Default: browserDecompress.

Since 0.1.0

redirectCount :: Int

How many redirects to follow when getting a resource. 0 means follow no redirects. Default value: 10.

Since 0.1.0

checkStatus :: Status -> ResponseHeaders -> CookieJar -> Maybe SomeException

Check the status code. Note that this will run after all redirects are performed. Default: return a StatusCodeException on non-2XX responses.

Since 0.1.0

responseTimeout :: Maybe Int

Number of microseconds to wait for a response. If Nothing, will wait indefinitely. Default: 5 seconds.

Since 0.1.0

getConnectionWrapper :: Maybe Int -> HttpException -> IO (ConnRelease, Connection, ManagedConn) -> IO (Maybe Int, (ConnRelease, Connection, ManagedConn))

Wraps the calls for getting new connections. This can be useful for instituting some kind of timeouts. The first argument is the value of responseTimeout. Second argument is the exception to be thrown on failure.

Default: If responseTimeout is Nothing, does nothing. Otherwise, institutes timeout, and returns remaining time for responseTimeout.

Since 0.1.0

cookieJar :: Maybe CookieJar

A user-defined cookie jar. If Nothing, no cookie handling will take place, "Cookie" headers in requestHeaders will be sent raw, and responseCookieJar will be empty.

Since 0.1.0

data ConnReuse Source

Constructors

Reuse 
DontReuse 

data ManagedConn Source

Constructors

Fresh 
Reused 

data Response body Source

A simple representation of the HTTP response.

Since 0.1.0

Constructors

Response 

Fields

responseStatus :: !Status

Status code of the response.

Since 0.1.0

responseVersion :: !HttpVersion

HTTP version used by the server.

Since 0.1.0

responseHeaders :: !ResponseHeaders

Response headers sent by the server.

Since 0.1.0

responseBody :: !body

Response body sent by the server.

Since 0.1.0

responseCookieJar :: !CookieJar

Cookies set on the client after interacting with the server. If cookies have been disabled by setting cookieJar to Nothing, then this will always be empty.

Since 0.1.0

responseClose' :: !ResponseClose

Releases any resource held by this response. If the response body has not been fully read yet, doing so after this call will likely be impossible.

Since 0.1.0

data Manager Source

Keeps track of open connections for keep-alive.

If possible, you should share a single Manager between multiple threads and requests.

Since 0.1.0

Constructors

Manager 

Fields

mConns :: !(IORef (Maybe (Map ConnKey (NonEmptyList Connection))))

Nothing indicates that the manager is closed.

mMaxConns :: !Int

This is a per-ConnKey value.

mResponseTimeout :: !(Maybe Int)

Copied from managerResponseTimeout

mRawConnection :: !(Maybe HostAddress -> String -> Int -> IO Connection)
 
mTlsConnection :: !(Maybe HostAddress -> String -> Int -> IO Connection)
 
mTlsProxyConnection :: !(ByteString -> (Connection -> IO ()) -> Maybe HostAddress -> String -> Int -> IO Connection)
 
mRetryableException :: !(SomeException -> Bool)
 
mWrapIOException :: !(forall a. IO a -> IO a)
 

data ManagerSettings Source

Settings for a Manager. Please use the defaultManagerSettings function and then modify individual settings. For more information, see http://www.yesodweb.com/book/settings-types.

Since 0.1.0

Constructors

ManagerSettings 

Fields

managerConnCount :: !Int

Number of connections to a single host to keep alive. Default: 10.

Since 0.1.0

managerRawConnection :: !(IO (Maybe HostAddress -> String -> Int -> IO Connection))

Create an insecure connection.

Since 0.1.0 FIXME in the future, combine managerTlsConnection and managerTlsProxyConnection

managerTlsConnection :: !(IO (Maybe HostAddress -> String -> Int -> IO Connection))

Create a TLS connection. Default behavior: throw an exception that TLS is not supported.

Since 0.1.0

managerTlsProxyConnection :: !(IO (ByteString -> (Connection -> IO ()) -> Maybe HostAddress -> String -> Int -> IO Connection))

Create a TLS proxy connection. Default behavior: throw an exception that TLS is not supported.

Since 0.2.2

managerResponseTimeout :: !(Maybe Int)

Default timeout (in microseconds) to be applied to requests which do not provide a timeout value.

Default is 5 seconds

Since 0.1.0

managerRetryableException :: !(SomeException -> Bool)

Exceptions for which we should retry our request if we were reusing an already open connection. In the case of IOExceptions, for example, we assume that the connection was closed on the server and therefore open a new one.

Since 0.1.0

managerWrapIOException :: !(forall a. IO a -> IO a)

Action wrapped around all attempted Requests, usually used to wrap up exceptions in library-specific types.

Default: wrap all IOExceptions in the InternalIOException constructor.

Since 0.1.0

data NonEmptyList a Source

Constructors

One !a !UTCTime 
Cons !a !Int !UTCTime !(NonEmptyList a) 

data ConnHost Source

Hostname or resolved host address.

data ConnKey Source

ConnKey consists of a hostname, a port and a Bool specifying whether to use SSL.

Constructors

ConnKey !ConnHost !Int !Bool 

Various utilities

hGetSome :: Handle -> Int -> IO ByteString

Like hGet, except that a shorter ByteString may be returned if there are not enough bytes immediately available to satisfy the whole request. hGetSome only blocks if there is no data available, and EOF has not yet been reached.

(<>) :: Monoid m => m -> m -> mSource

hasNoBodySource

Arguments

:: ByteString

request method

-> Int

status code

-> Bool 

fromStrict :: ByteString -> ByteString

O(1) Convert a strict ByteString into a lazy ByteString.