Copyright | © 2020 Francesco Ariis |
---|---|
License | GPLv3 (see LICENSE file) |
Maintainer | Francesco Ariis <fa-ml@ariis.it> |
Stability | provisional |
Portability | portable |
Safe Haskell | Safe-Inferred |
Language | Haskell2010 |
Text.HTML.Scalpel.Core scrapers for DuckDuckGo and Google search engines.
Example:
import Text.HTML.Scalpel.Search import Text.HTML.Scalpel -- from package `scalpel` url = buildGoogleURL "Gugliemo Oberdan" sa = scrapeURL (show url) (googleScraper :: Scraper String SearchResult) main = sa >>= print
Synopsis
- type Query = Text
- buildDuckduckgoURL :: Query -> URI
- buildGoogleURL :: Query -> URI
- url2Text :: URI -> Text
- newtype SearchResult = SearchResult {}
- duckduckgoScraper :: (StringLike str, Monad m) => ScraperT str m SearchResult
- googleScraper :: (Show str, StringLike str, Monad m) => ScraperT str m SearchResult
URL building
Check URI
to modify standard queries.
Scrapers
newtype SearchResult Source #
Instances
Show SearchResult Source # | |
Defined in Text.HTML.Scalpel.Search showsPrec :: Int -> SearchResult -> ShowS # show :: SearchResult -> String # showList :: [SearchResult] -> ShowS # | |
Eq SearchResult Source # | |
Defined in Text.HTML.Scalpel.Search (==) :: SearchResult -> SearchResult -> Bool # (/=) :: SearchResult -> SearchResult -> Bool # |
duckduckgoScraper :: (StringLike str, Monad m) => ScraperT str m SearchResult Source #
Scrape a DuckDuckGo search page for result titles and links. DuckDuckGo is not fond of bots downloading their pages and will display an error if searches are too frequent (≳ 1 search per second).
googleScraper :: (Show str, StringLike str, Monad m) => ScraperT str m SearchResult Source #
Scrape a Google search page for result titles and links.