The shpider package
Shpider is a web automation library for Haskell. It allows you to quickly write crawlers, and for simple cases ( like following links ) even without reading the page source.
It has useful features such as turning relative links from a page into absolute links, options to authorize transactions only on a given domain, and the option to only download html documents.
It also provides a nice syntax for filling out forms.
runShpider $ do download "http://apage.com" theForm : _ <- getFormsByAction "http://anotherpage.com" sendForm $ fillOutForm theForm $ pairs $ do "occupation" =: "unemployed Haskell programmer" "location" =: "mother's house"
Shpider contains a patched version of the curl package, to fix cookie handling. The curl licence is therefore distributed with this package.
|Versions||0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.6, 0.0.7, 0.1.0, 0.1.1, 0.2, 0.2.1.1|
|Dependencies||base, bytestring, containers, mtl, regex-posix, tagsoup, tagsoup-parsec, url (==2) [details]|
|Maintainer||Johnny Morrice <firstname.lastname@example.org>|
|Uploaded||Fri Mar 20 18:38:02 UTC 2009 by JohnnyMorrice|
|Downloads||2169 total (22 in the last 30 days)|
|Status||Docs uploaded by user
Build status unknown [no reports yet]
For package maintainers and hackage trustees