Follow.Fetchers.WebScraping.Internal

This module contains the inner wiring for the scraping fetching strategy.

Documentation

data Selector

Data type with the selectors to use when scraping each Entry item.

Constructors

Selector

Fields

• selURI :: Maybe SelectorItem
• selGUID :: Maybe SelectorItem
• selTitle :: Maybe SelectorItem
• selDescription :: Maybe SelectorItem
• selAuthor :: Maybe SelectorItem
• selPublishDate :: Maybe SelectorItem

Instances

Eq Selector

Defined in Follow.Fetchers.WebScraping.Internal

Methods

(==) :: Selector -> Selector -> Bool

(/=) :: Selector -> Selector -> Bool

Show Selector

Defined in Follow.Fetchers.WebScraping.Internal

Methods

showsPrec :: Int -> Selector -> ShowS

show :: Selector -> String

showList :: [Selector] -> ShowS

FromJSON Selector

  uri: # See SelectorItem instance
  title: null
  description: null
  guid: null
  author: null
  publish_date: null
 

Defined in Follow.Parser

Methods

parseJSON :: Value -> Parser Selector

parseJSONList :: Value -> Parser [Selector]

data SelectorItem

Selector to use when scraping an Entry item.

Constructors

InnerText CSSSelector

This selector will take the inner text immediately descendant of a tag selected with given css selector.

Attr CSSSelector HTMLAttribute

This selector will take the value of given argument in the tag matched by given css selector.

Instances

Eq SelectorItem

Defined in Follow.Fetchers.WebScraping.Internal

Methods

(==) :: SelectorItem -> SelectorItem -> Bool

(/=) :: SelectorItem -> SelectorItem -> Bool

Show SelectorItem

Defined in Follow.Fetchers.WebScraping.Internal

Methods

showsPrec :: Int -> SelectorItem -> ShowS

show :: SelectorItem -> String

showList :: [SelectorItem] -> ShowS

FromJSON SelectorItem

  type: text
  options:
    css: .selector
  

or

  type: attr
  options:
    css: .link
    name: href
  

Defined in Follow.Parser

Methods

parseJSON :: Value -> Parser SelectorItem

parseJSONList :: Value -> Parser [SelectorItem]

type CSSSelector = Text

A CSS2 selector.

type HTMLAttribute = Text

An HTML attribute name.

htmlToEntries :: ByteString -> Selector -> IO [Entry]

Converts a bytestring with HTML content to a list of entries, scraping entry items using given selector. The return type is wrapped within an IO because of the underlying vendor API.