wreq-0.4.1.0: An easy-to-use HTTP client library.

Copyright(c) 2014 Bryan O'Sullivan
LicenseBSD-style
Maintainerbos@serpentine.com
Stabilityexperimental
PortabilityGHC
Safe HaskellNone
LanguageHaskell98

Network.Wreq

Contents

Description

A library for client-side HTTP requests, focused on ease of use.

When reading the examples in this module, you should assume the following environment:

-- Make it easy to write literal ByteString and Text values.
{-# LANGUAGE OverloadedStrings #-}

-- Our handy module.
import Network.Wreq

-- Operators such as (&) and (.~).
import Control.Lens

-- Conversion of Haskell values to JSON.
import Data.Aeson (toJSON)

-- Easy traversal of JSON data.
import Data.Aeson.Lens (key, nth)

There exist some less frequently used lenses that are not exported from this module; these can be found in Network.Wreq.Lens.

Synopsis

HTTP verbs

Sessions

The basic HTTP functions (get, post, and so on) in this module have a few key drawbacks:

  • If several requests go to the same server, there is no reuse of TCP connections.
  • There is no management of cookies across multiple requests.

This makes these functions inefficient and verbose for many common uses. For greater efficiency, use the Network.Wreq.Session module.

GET

get :: String -> IO (Response ByteString) Source

Issue a GET request.

Example:

get "http://httpbin.org/get"
 
>>> r <- get "http://httpbin.org/get"
>>> r ^. responseStatus . statusCode
200

getWith :: Options -> String -> IO (Response ByteString) Source

Issue a GET request, using the supplied Options.

Example:

let opts = defaults & param "foo" .~ ["bar"]
getWith opts "http://httpbin.org/get"
 
>>> let opts = defaults & param "foo" .~ ["bar"]
>>> r <- getWith opts "http://httpbin.org/get"
>>> r ^? responseBody . key "url"
Just (String "http://httpbin.org/get?foo=bar")

POST

The Postable class determines which Haskell types can be used as POST payloads.

Part and [Part] give a request body with a Content-Type of multipart/form-data. Constructor functions include partText and partFile.

>>> r <- post "http://httpbin.org/post" (partText "hello" "world")
>>> r ^? responseBody . key "form" . key "hello"
Just (String "world")

(ByteString, ByteString) and FormParam (and lists of each) give a request body with a Content-Type of application/x-www-form-urlencoded. The easiest way to use this is via the (:=) constructor.

>>> r <- post "http://httpbin.org/post" ["num" := 31337, "str" := "foo"]
>>> r ^? responseBody . key "form" . key "num"
Just (String "31337")

The "magical" type conversion on the right-hand side of := above is due to the FormValue class. This package provides sensible instances for the standard string and number types.

The Value type gives a JSON request body with a Content-Type of application/json. Any instance of ToJSON can of course be converted to a Value using toJSON.

>>> r <- post "http://httpbin.org/post" (toJSON [1,2,3])
>>> r ^? responseBody . key "json" . nth 0
Just (Number 1.0)

post :: Postable a => String -> a -> IO (Response ByteString) Source

Issue a POST request.

Example:

post "http://httpbin.org/post" (toJSON [1,2,3])
 
>>> r <- post "http://httpbin.org/post" (toJSON [1,2,3])
>>> r ^? responseBody . key "json" . nth 2
Just (Number 3.0)

postWith :: Postable a => Options -> String -> a -> IO (Response ByteString) Source

Issue a POST request, using the supplied Options.

Example:

let opts = defaults & param "foo" .~ ["bar"]
postWith opts "http://httpbin.org/post" (toJSON [1,2,3])
 
>>> let opts = defaults & param "foo" .~ ["bar"]
>>> r <- postWith opts "http://httpbin.org/post" (toJSON [1,2,3])
>>> r ^? responseBody . key "url"
Just (String "http://httpbin.org/post?foo=bar")

HEAD

head_ :: String -> IO (Response ()) Source

Issue a HEAD request.

Example:

head_ "http://httpbin.org/get"
 
>>> r <- head_ "http://httpbin.org/get"
>>> r ^? responseHeader "Content-Type"
Just "application/json"

headWith :: Options -> String -> IO (Response ()) Source

Issue a HEAD request, using the supplied Options.

Example:

let opts = defaults & param "foo" .~ ["bar"]
headWith opts "http://httpbin.org/get"
 
>>> let opts = defaults & param "foo" .~ ["bar"]
>>> r <- headWith opts "http://httpbin.org/get"
>>> r ^? responseHeader "Connection"
Just "keep-alive"

OPTIONS

options :: String -> IO (Response ()) Source

Issue an OPTIONS request.

Example:

options "http://httpbin.org/get"
 

See atto for a more complex worked example.

optionsWith :: Options -> String -> IO (Response ()) Source

Issue an OPTIONS request, using the supplied Options.

Example:

let opts = defaults & param "foo" .~ ["bar"]
optionsWith opts "http://httpbin.org/get"
 

PUT

put :: Putable a => String -> a -> IO (Response ByteString) Source

Issue a PUT request.

putWith :: Putable a => Options -> String -> a -> IO (Response ByteString) Source

Issue a PUT request, using the supplied Options.

DELETE

delete :: String -> IO (Response ByteString) Source

Issue a DELETE request.

Example:

delete "http://httpbin.org/delete"
 
>>> r <- delete "http://httpbin.org/delete"
>>> r ^. responseStatus . statusCode
200

deleteWith :: Options -> String -> IO (Response ByteString) Source

Issue a DELETE request, using the supplied Options.

Example:

let opts = defaults & redirects .~ 0
deleteWith opts "http://httpbin.org/delete"
 
>>> let opts = defaults & redirects .~ 0
>>> r <- deleteWith opts "http://httpbin.org/delete"
>>> r ^. responseStatus . statusCode
200

Custom Method

customMethod :: String -> String -> IO (Response ByteString) Source

Issue a custom-method request

Example: customMethod "PATCH" "http://httpbin.org/patch"

>>> r <- customMethod "PATCH" "http://httpbin.org/patch"
>>> r ^. responseStatus . statusCode
200

customMethodWith :: String -> Options -> String -> IO (Response ByteString) Source

Issue a custom request method request, using the supplied Options.

Example:

let opts = defaults & redirects .~ 0
customMethodWith "PATCH" opts "http://httpbin.org/patch"
 
>>> let opts = defaults & redirects .~ 0
>>> r <- customMethodWith "PATCH" opts "http://httpbin.org/patch"
>>> r ^. responseStatus . statusCode
200

Custom Payload Method

customPayloadMethod :: Postable a => String -> String -> a -> IO (Response ByteString) Source

Issue a custom-method request with a payload

customPayloadMethodWith :: Postable a => String -> Options -> String -> a -> IO (Response ByteString) Source

Issue a custom-method request with a payload, using the supplied Options.

Incremental consumption of responses

GET

foldGet :: (a -> ByteString -> IO a) -> a -> String -> IO a Source

foldGetWith :: Options -> (a -> ByteString -> IO a) -> a -> String -> IO a Source

Configuration

data Options Source

Options for configuring a client.

Instances

manager :: Lens' Options (Either ManagerSettings Manager) Source

A lens onto configuration of the connection manager provided by the http-client package.

In this example, we enable the use of OpenSSL for (hopefully) secure connections:

import OpenSSL.Session (context)
import Network.HTTP.Client.OpenSSL

let opts = defaults & manager .~ Left (opensslManagerSettings context)
withOpenSSL $
  getWith opts "https://httpbin.org/get"
 

In this example, we also set the response timeout to 10000 microseconds:

import OpenSSL.Session (context)
import Network.HTTP.Client.OpenSSL
import Network.HTTP.Client (defaultManagerSettings, managerResponseTimeout)

let opts = defaults & manager .~ Left (opensslManagerSettings context)
                    & manager .~ Left (defaultManagerSettings { managerResponseTimeout = Just 10000 } )

withOpenSSL $
  getWith opts "https://httpbin.org/get"
 

header :: HeaderName -> Lens' Options [ByteString] Source

A lens onto all headers with the given name (there can legitimately be zero or more).

Example:

let opts = defaults & header "Accept" .~ ["*/*"]
getWith opts "http://httpbin.org/get"
 

param :: Text -> Lens' Options [Text] Source

A lens onto all query parameters with the given name (there can legitimately be zero or more).

In this example, we construct the query URL "http://httpbin.org/get?foo=bar&foo=quux".

let opts = defaults & param "foo" .~ ["bar", "quux"]
getWith opts "http://httpbin.org/get"
 

redirects :: Lens' Options Int Source

A lens onto the maximum number of redirects that will be followed before an exception is thrown.

In this example, a HttpException will be thrown with a TooManyRedirects constructor, because the maximum number of redirects allowed will be exceeded.

let opts = defaults & redirects .~ 3
getWith opts "http://httpbin.org/redirect/5"
 

headers :: Lens' Options [Header] Source

A lens onto all headers (there can legitimately be zero or more).

In this example, we print all the headers sent by default with every request.

print (defaults ^. headers)
 

params :: Lens' Options [(Text, Text)] Source

A lens onto all query parameters.

cookie :: ByteString -> Traversal' Options Cookie Source

A traversal onto the cookie with the given name, if one exists.

N.B. This is an "illegal" Traversal': we can change the cookieName of the associated Cookie so that it differs from the name provided to this function.

cookies :: Lens' Options (Maybe CookieJar) Source

A lens onto all cookies.

checkStatus :: Lens' Options (Maybe StatusChecker) Source

A lens to get the optional status check function

Authentication

Do not use HTTP authentication unless you are using TLS encryption. These authentication tokens can easily be captured and reused by an attacker if transmitted in the clear.

data Auth Source

Supported authentication types.

Do not use HTTP authentication unless you are using TLS encryption. These authentication tokens can easily be captured and reused by an attacker if transmitted in the clear.

data AWSAuthVersion Source

Constructors

AWSv4

AWS request signing version 4

auth :: Lens' Options (Maybe Auth) Source

A lens onto request authentication.

Example (note the use of TLS):

let opts = defaults & auth ?~ basicAuth "user" "pass"
getWith opts "https://httpbin.org/basic-auth/user/pass"
 

basicAuth Source

Arguments

:: ByteString

Username.

-> ByteString

Password.

-> Auth 

Basic authentication. This consists of a plain username and password.

Example (note the use of TLS):

let opts = defaults & auth ?~ basicAuth "user" "pass"
getWith opts "https://httpbin.org/basic-auth/user/pass"
 

Note here the use of the ?~ setter to turn an Auth into a Maybe Auth, to make the type of the RHS compatible with the auth lens.

>>> let opts = defaults & auth ?~ basicAuth "user" "pass"
>>> r <- getWith opts "https://httpbin.org/basic-auth/user/pass"
>>> r ^? responseBody . key "authenticated"
Just (Bool True)

oauth1Auth Source

Arguments

:: ByteString

Consumer token

-> ByteString

Consumer secret

-> ByteString

OAuth token

-> ByteString

OAuth token secret

-> Auth 

OAuth1 authentication. This consists of a consumer token, a consumer secret, a token and a token secret

oauth2Bearer :: ByteString -> Auth Source

An OAuth2 bearer token. This is treated by many services as the equivalent of a username and password.

Example (note the use of TLS):

let opts = defaults & auth ?~ oauth2Bearer "1234abcd"
getWith opts "https://public-api.wordpress.com/rest/v1/me/"
 

oauth2Token :: ByteString -> Auth Source

A not-quite-standard OAuth2 bearer token (that seems to be used only by GitHub). This will be treated by whatever services accept it as the equivalent of a username and password.

Example (note the use of TLS):

let opts = defaults & auth ?~ oauth2Token "abcd1234"
getWith opts "https://api.github.com/user"
 

awsAuth :: AWSAuthVersion -> ByteString -> ByteString -> Auth Source

AWS v4 request signature.

Example (note the use of TLS):

let opts = defaults & auth ?~ 'awsAuth AWSv4' "key" "secret"
getWith opts "https://dynamodb.us-west-2.amazonaws.com"
 

Proxy settings

data Proxy :: *

Define a HTTP proxy, consisting of a hostname and port number.

Constructors

Proxy ByteString Int 

proxy :: Lens' Options (Maybe Proxy) Source

A lens onto proxy configuration.

Example:

let opts = defaults & proxy ?~ httpProxy "localhost" 8000
getWith opts "http://httpbin.org/get"
 

Note here the use of the ?~ setter to turn a Proxy into a Maybe Proxy, to make the type of the RHS compatible with the proxy lens.

httpProxy :: ByteString -> Int -> Proxy Source

Proxy configuration.

Example:

let opts = defaults & proxy ?~ httpProxy "localhost" 8000
getWith opts "http://httpbin.org/get"
 

Note here the use of the ?~ setter to turn a Proxy into a Maybe Proxy, to make the type of the RHS compatible with the proxy lens.

Using a manager with defaults

withManager :: (Options -> IO a) -> IO a Source

Payloads for POST and PUT

data Payload where Source

A product type for representing more complex payload types.

Constructors

Raw :: ContentType -> RequestBody -> Payload 

URL-encoded form data

data FormParam where Source

A key/value pair for an application/x-www-form-urlencoded POST request body.

Constructors

(:=) :: FormValue v => ByteString -> v -> FormParam infixr 3 

Instances

class FormValue a Source

A type that can be rendered as the value portion of a key/value pair for use in an application/x-www-form-urlencoded POST body. Intended for use with the FormParam type.

The instances for String, strict Text, and lazy Text are all encoded using UTF-8 before being URL-encoded.

The instance for Maybe gives an empty string on Nothing, and otherwise uses the contained type's instance.

Minimal complete definition

renderFormValue

Multipart form data

data Part :: *

A single part of a multipart message.

Instances

partName :: Lens' Part Text Source

A lens onto the name of the input element associated with part of a multipart form upload.

partFileName :: Lens' Part (Maybe String) Source

A lens onto the filename associated with part of a multipart form upload.

partContentType :: Traversal' Part (Maybe MimeType) Source

A lens onto the content-type associated with part of a multipart form upload.

partGetBody :: Lens' Part (IO RequestBody) Source

A lens onto the code that fetches the data associated with part of a multipart form upload.

Smart constructors

partBS

Arguments

:: Text

Name of the corresponding <input>.

-> ByteString

The body for this Part.

-> Part 

Make a Part whose content is a strict ByteString.

The Part does not have a file name or content type associated with it.

partLBS

Arguments

:: Text

Name of the corresponding <input>.

-> ByteString

The body for this Part.

-> Part 

Make a Part whose content is a lazy ByteString.

The Part does not have a file name or content type associated with it.

partText Source

Arguments

:: Text

Name of the corresponding <input>.

-> Text

The body for this Part.

-> Part 

Make a Part whose content is a strict Text, encoded as UTF-8.

The Part does not have a file name or content type associated with it.

partString Source

Arguments

:: Text

Name of the corresponding <input>.

-> String

The body for this Part.

-> Part 

Make a Part whose content is a String, encoded as UTF-8.

The Part does not have a file name or content type associated with it.

partFile

Arguments

:: Text

Name of the corresponding <input>.

-> FilePath

The name of the local file to upload.

-> Part 

Make a Part from a file.

The entire file will reside in memory at once. If you want constant memory usage, use partFileSource.

The FilePath supplied will be used as the file name of the Part. If you do not want to reveal this name to the server, you must remove it prior to uploading.

The Part does not have a content type associated with it.

partFileSource

Arguments

:: Text

Name of the corresponding <input>.

-> FilePath

The name of the local file to upload.

-> Part 

Stream a Part from a file.

The FilePath supplied will be used as the file name of the Part. If you do not want to reveal this name to the server, you must remove it prior to uploading.

The Part does not have a content type associated with it.

Responses

data Response body :: * -> *

A simple representation of the HTTP response.

Since 0.1.0

Instances

responseBody :: Lens (Response body0) (Response body1) body0 body1 Source

A lens onto the body of a response.

r <- get "http://httpbin.org/get"
print (r ^. responseBody)
 

responseHeader Source

Arguments

:: HeaderName

Header name to match.

-> Traversal' (Response body) ByteString 

A lens onto all matching named headers in an HTTP response.

To access exactly one header (the result will be the empty string if there is no match), use the (^.) operator.

r <- get "http://httpbin.org/get"
print (r ^. responseHeader "Content-Type")
 

To access at most one header (the result will be Nothing if there is no match), use the (^?) operator.

r <- get "http://httpbin.org/get"
print (r ^? responseHeader "Content-Transfer-Encoding")
 

To access all (zero or more) matching headers, use the (^..) operator.

r <- get "http://httpbin.org/get"
print (r ^.. responseHeader "Set-Cookie")
 

responseLink Source

Arguments

:: ByteString

Parameter name to match.

-> ByteString

Parameter value to match.

-> Fold (Response body) Link 

A fold over Link headers, matching on both parameter name and value.

For example, here is a Link header returned by the GitHub search API.

Link:
  <https://api.github.com/search/code?q=addClass+user%3Amozilla&page=2>; rel="next",
  <https://api.github.com/search/code?q=addClass+user%3Amozilla&page=34>; rel="last"

And here is an example of how we can retrieve the URL for the next link programatically.

r <- get "https://api.github.com/search/code?q=addClass+user:mozilla"
print (r ^? responseLink "rel" "next" . linkURL)
 

responseCookie Source

Arguments

:: ByteString

Name of cookie to match.

-> Fold (Response body) Cookie 

A fold over any cookies that match the given name.

r <- get "http://www.nytimes.com/"
print (r ^? responseCookie "RMID")
 

responseHeaders :: Lens' (Response body) ResponseHeaders Source

A lens onto all headers in an HTTP response.

responseCookieJar :: Lens' (Response body) CookieJar Source

A lens onto all cookies set in the response.

responseStatus :: Lens' (Response body) Status Source

A lens onto the status of an HTTP response.

data Status :: *

HTTP Status.

Only the statusCode is used for comparisons.

Please use mkStatus to create status codes from code and message, or the Enum instance or the status code constants (like ok200). There might be additional record members in the future.

Note that the Show instance is only for debugging.

statusCode :: Lens' Status Int Source

A lens onto the numeric identifier of an HTTP status.

statusMessage :: Lens' Status ByteString Source

A lens onto the textual description of an HTTP status.

Link headers

data Link Source

An element of a Link header.

linkURL :: Lens' Link ByteString Source

A lens onto the URL portion of a Link element.

linkParams :: Lens' Link [(ByteString, ByteString)] Source

A lens onto the parameters of a Link element.

Decoding responses

data JSONError Source

The error type used by asJSON and asValue if a failure occurs when parsing a response body as JSON.

Constructors

JSONError String 

asJSON :: (MonadThrow m, FromJSON a) => Response ByteString -> m (Response a) Source

Convert the body of an HTTP response from JSON to a suitable Haskell type.

In this example, we use asJSON in the IO monad, where it will throw a JSONError exception if conversion to the desired type fails.

 {-# LANGUAGE DeriveGeneric #-}
import GHC.Generics (Generic)

 {- This Haskell type corresponds to the structure of a
   response body from httpbin.org. -}

data GetBody = GetBody {
    headers :: Map Text Text
  , args :: Map Text Text
  , origin :: Text
  , url :: Text
  } deriving (Show, Generic)

 -- Get GHC to derive a FromJSON instance for us.
instance FromJSON GetBody

 {- The fact that we want a GetBody below will be inferred by our
   use of the "headers" accessor function. -}

foo = do
  r <- asJSON =<< get "http://httpbin.org/get"
  print (headers (r ^. responseBody))
 

If we use asJSON in the Either monad, it will return Left with a JSONError payload if conversion fails, and Right with a Response whose responseBody is the converted value on success.

asValue :: MonadThrow m => Response ByteString -> m (Response Value) Source

Convert the body of an HTTP response from JSON to a Value.

In this example, we use asValue in the IO monad, where it will throw a JSONError exception if the conversion to Value fails.

foo = do
  r <- asValue =<< get "http://httpbin.org/get"
  print (r ^? responseBody . key "headers" . key "User-Agent")
 

Cookies

These are only the most frequently-used cookie-related lenses. See Network.Wreq.Lens for the full accounting of them all.

data Cookie :: *

cookieName :: Lens' Cookie ByteString Source

A lens onto the name of a cookie.

cookieValue :: Lens' Cookie ByteString Source

A lens onto the value of a cookie.

cookieExpiryTime :: Lens' Cookie UTCTime Source

A lens onto the expiry time of a cookie.

cookieDomain :: Lens' Cookie ByteString Source

A lens onto the domain of a cookie.

cookiePath :: Lens' Cookie ByteString Source

A lens onto the path of a cookie.

Parsing responses

atto :: Parser a -> Fold ByteString a Source

Turn an attoparsec Parser into a Fold.

Both headers and bodies can contain complicated data that we may need to parse.

Example: when responding to an OPTIONS request, a server may return the list of verbs it supports in any order, up to and including changing the order on every request (which httpbin.org /actually does/!). To deal with this possibility, we parse the list, then sort it.

>>> import Data.Attoparsec.ByteString.Char8 as A
>>> import Data.List (sort)
>>> 
>>> let comma = skipSpace >> "," >> skipSpace
>>> let verbs = A.takeWhile isAlpha_ascii `sepBy` comma
>>> 
>>> r <- options "http://httpbin.org/get"
>>> r ^. responseHeader "Allow" . atto verbs . to sort
["GET","HEAD","OPTIONS"]

atto_ :: Parser a -> Fold ByteString a Source

The same as atto, but ensures that the parser consumes the entire input.

Equivalent to:

atto_ myParser = atto (myParser <* endOfInput)