scalpel-search-0.1.0.2: scalpel scrapers for search engines
Copyright© 2020 Francesco Ariis
LicenseGPLv3 (see LICENSE file)
MaintainerFrancesco Ariis <fa-ml@ariis.it>
Stabilityprovisional
Portabilityportable
Safe HaskellSafe-Inferred
LanguageHaskell2010

Text.HTML.Scalpel.Search

Description

Text.HTML.Scalpel.Core scrapers for DuckDuckGo and Google search engines.

Example:

import Text.HTML.Scalpel.Search
import Text.HTML.Scalpel          -- from package `scalpel`

url = buildGoogleURL "Gugliemo Oberdan"
sa = scrapeURL (show url) (googleScraper :: Scraper String SearchResult)
main = sa >>= print
Synopsis

URL building

Check URI to modify standard queries.

buildDuckduckgoURL :: Query -> URI Source #

Build a DuckDuckGo URL from a Query string.

buildGoogleURL :: Query -> URI Source #

Build a Googlr URL from a Query string.

url2Text :: URI -> Text Source #

Convert URI to Text. If you prefer String, just use show.

Scrapers

newtype SearchResult Source #

Constructors

SearchResult 

Fields

Instances

Instances details
Show SearchResult Source # 
Instance details

Defined in Text.HTML.Scalpel.Search

Eq SearchResult Source # 
Instance details

Defined in Text.HTML.Scalpel.Search

duckduckgoScraper :: (StringLike str, Monad m) => ScraperT str m SearchResult Source #

Scrape a DuckDuckGo search page for result titles and links. DuckDuckGo is not fond of bots downloading their pages and will display an error if searches are too frequent (≳ 1 search per second).

googleScraper :: (Show str, StringLike str, Monad m) => ScraperT str m SearchResult Source #

Scrape a Google search page for result titles and links.