The shpider package

[ Tags: bsd3, library, web ] [ Propose Tags ]

Shpider is a web automation library for Haskell. It allows you to quickly write crawlers, and for simple cases ( like following links ) even without reading the page source.

It has useful features such as turning relative links from a page into absolute links, options to authorize transactions only on a given domain, and the option to only download html documents.

It also provides a nice syntax for filling out forms.

An example:

runShpider $ do
     download ""
     theForm : _ <- getFormsByAction ""
     sendForm $ fillOutForm theForm $ pairs $ do
           "occupation" =: "unemployed Haskell programmer"
           "location" =: "mother's house"

Shpider contains a patched version of the curl package, to fix cookie handling. The curl licence is therefore distributed with this package.


Versions 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.6, 0.0.7, 0.1.0, 0.1.1, 0.2,
Dependencies base, bytestring, containers, mtl, regex-posix, tagsoup, tagsoup-parsec, url (==2) [details]
License BSD3
Author Johnny Morrice
Maintainer Johnny Morrice <>
Category Web
Home page
Uploaded Sun Mar 22 11:35:55 UTC 2009 by JohnnyMorrice
Distributions NixOS:
Downloads 3641 total (48 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Docs uploaded by user
Build status unknown [no reports yet]
Hackage Matrix CI




Maintainer's Corner

For package maintainers and hackage trustees