Text.HTML.Scalpel.Search

Text.HTML.Scalpel.Core scrapers for DuckDuckGo and Google search engines.

Example:

import Text.HTML.Scalpel.Search
import Text.HTML.Scalpel          -- from package `scalpel`

url = buildGoogleURL "Gugliemo Oberdan"
sa = scrapeURL (show url) (googleScraper :: Scraper String SearchResult)
main = sa >>= print

URL building

Check URI to modify standard queries.

type Query = Text

buildDuckduckgoURL :: Query -> URI

Build a DuckDuckGo URL from a Query string.

buildGoogleURL :: Query -> URI

Build a Googlr URL from a Query string.

url2Text :: URI -> Text

Convert URI to Text. If you prefer String, just use show.

Scrapers

newtype SearchResult

Constructors

SearchResult

Fields

• unwrapSR :: [(Title, URI)]

Instances

Show SearchResult

Defined in Text.HTML.Scalpel.Search

Methods

showsPrec :: Int -> SearchResult -> ShowS

show :: SearchResult -> String

showList :: [SearchResult] -> ShowS

Eq SearchResult

Defined in Text.HTML.Scalpel.Search

Methods

(==) :: SearchResult -> SearchResult -> Bool

(/=) :: SearchResult -> SearchResult -> Bool

duckduckgoScraper :: (StringLike str, Monad m) => ScraperT str m SearchResult

Scrape a DuckDuckGo search page for result titles and links. DuckDuckGo is not fond of bots downloading their pages and will display an error if searches are too frequent (≳ 1 search per second).

googleScraper :: (Show str, StringLike str, Monad m) => ScraperT str m SearchResult

Scrape a Google search page for result titles and links.