The shpider package

[Tags:bsd3, library]

Shpider is a web automation library for Haskell. It allows you to quickly write crawlers, and for simple cases ( like following links ) even without reading the page source.

It has useful features such as turning relative links from a page into absolute links, options to authorize transactions only on a given domain, and the option to only download html documents.

It also provides a nice syntax for filling out forms.

An example:

 runShpider $ do
      download ""
      theForm : _ <- getFormsByAction ""
      sendForm $ fillOutForm theForm $ pairs $ do
            "occupation" =: "unemployed Haskell programmer"
            "location" =: "mother's house"

Shpider contains a patched version of the curl package ( the original package's garbage-collection caused non-deterministic behaviour ). The curl licence is therefore distributed with this package.


Versions 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.6, 0.0.7, 0.1.0, 0.1.1, 0.2,
Dependencies base (<5), bytestring, containers, mtl, regex-posix, tagsoup, tagsoup-parsec, url (>=2) [details]
License BSD3
Author Johnny Morrice
Maintainer Johnny Morrice <>
Stability Unknown
Category Web
Home page
Uploaded Thu Aug 6 16:22:24 UTC 2009 by JohnnyMorrice
Distributions NixOS:
Downloads 2051 total (17 in the last 30 days)
0 []
Status Docs uploaded by user
Build status unknown [no reports yet]




Maintainer's Corner

For package maintainers and hackage trustees