Network.Shpider.State

This module describes the state of shpider computations, and provides a monad transformer over it.

Documentation

module Control.Monad.State

data ShpiderState

The shpider state holds all the options for shpider transactions, the current page and all the CurlOptions used when calling curl.

Constructors

SS

Fields

htmlOnlyDownloads :: Bool

startPage :: String

dontLeaveDomain :: Bool

curlOpts :: [CurlOption]

currentPage :: Page

visited :: Maybe [String]

downloadThrottle :: Maybe Int

Whether to wait at least N micro-seconds between downloads or form submissions. Defaults to Nothing.

lastDownloadTime :: Maybe UTCTime

Instances

Show ShpiderState

data Page

The Page datatype. Holds Links, Forms, the parsed [ Tag ], the page source, and the page's absolute URL.

Constructors

Page

Fields

links :: [Link]

forms :: [Form]

tags :: [Tag String]

source :: String

addr :: String

Instances

Show Page

type Shpider = StateT ShpiderState IO

The type of Shpider computations. A state transformer over ShpiderState and IO.

emptyPage :: Page

An empty page, containing no information.

runShpider :: Shpider a -> IO a

Run a Shpider computation, returning the result.

runShpiderSt :: Shpider a -> IO (a, ShpiderState)

Run a Shpider computation, returning the result with the state.