5.YL      !"#$%&'()*+,-./0123456789:;<=>?@ A B C D E F G H I J K  SafeLMLMLM SafeNOPQRSNOPQRNOPQRSSafeSafeSafe BAdds a prefix to a relative crawl action to get an absolute one. |      Safe    NoneTUTUTUSafe   Safe A crawl directive takes a content of a web page and produces crawl actions for links/forms to follow. The general idea is to specify a list of operations that in theory produces a dynamically collected tree of requests which leaves are either dead ends or end results.Additional, logical branching/combination of Directives is possible with: * Alternatives - evaluate both Directives in order. * Restart - evaluate completely new initial action & chain if the previous combo does not produce end results..access content to find absolute follow-up urls0as simple, but found relative urls are completed-as simple, but with access to complete result (wait additional seconds before executing!5if given directive yields no results use add. retries"6fallback to second argument if first yields no results#=the possibility to start a new chain (when using alternative)$1not crawling anything, just a blacklisting option%chaining of directives  !"#$%  !"#$%  !"#$%  !"#$%Safe(name of crawl run)starting point*7list of operations sequentially on all previous results+Kstore the content of a single result (the first) of the last operation step,Hstore the url of a single result (the first) of the last operation step&'()*+,&'()*+,&'()*+,&'()*+,SafeV(name, content, attributes)2TContent of the tags up to the first child tag (as simplification) - with attributes.!-./V01W23456789:;<X=YZ>?[\]^_`abc-./0123456789:;<=>?2389:;<7>=?-./41506-./V01W23456789:;<X=YZ>?[\]^_`abc Safe@Adefg@A@A@AdefgNoneh=Processes one step of a crawl chain: does the actual loading.itUsed for preparation of integration tests: additionally stores the crawl result using the given file name strategy.jkhilmjkhijkhilm NonentMake a unique name for a crawl action - prefix is used to specify the target folder including a specific test prefix opqBCDEnFrsBCDEFBCDEFopqBCDEnFrsNonetLazily evaluate each action in the sequence from left to right, and collect the results. PS: also playing around with an additional concat before returning Guvwtxyz{|}~  !"#$%G| Guvwtxyz{|}~ NoneJXReturns only the first result of a completely matching branch of the crawling directive.KiReturns all possible results of the craling directive - meant to be used with lazyness in mind as needed.HIJKGHIJKJKHIGHIJK !"#$%&''()*+,-./01234556789:;<=>?@ABCDEFGHIJKLM N O P Q R S TU V W X Y Z [ \ \ ] ^ _ `abcdefghijklmnop q r s tuvwxyz { | | } ~  crawl_ICiXsaNpcl2FBHyUb9Qm0gNetwork.CrawlChain.StoringNetwork.URI.UtilNetwork.CrawlChain.CrawlAction'Network.CrawlChain.DirectiveChainResultNetwork.CrawlChain.CrawlResult!Network.CrawlChain.CrawlDirective%Network.CrawlChain.CrawlingParameters"Text.HTML.CrawlChain.HtmlFiltering!Network.CrawlChain.BasicTemplates"Network.CrawlChain.CrawlingContextNetwork.CrawlChainNetwork.CrawlChain.UtilNetwork.CrawlChain.ReportNetwork.CrawlChain.DownloadingNetwork.CrawlChain.CrawlingNetwork.CrawlChain.CrawlChains buildCurlCmdbuildAndCreateTargetDirtoURI CrawlAction GetRequest PostRequest PostParamsPostType UndefinedPostFormPostAJAXcrawlUrl addUrlPrefixDirectiveChainResult resultHistory lastResultshowResultPathextractFirstResultCrawlingResultStatus CrawlingOkCrawlingRedirectCrawlingFailed CrawlResultcrawlingActioncrawlingContentcrawlingResultStatusCrawlDirectiveSimpleDirectiveRelativeDirectiveFollowUpDirectiveDelayDirectiveRetryDirectiveAlternativeDirectiveRestartChainDirectiveGuardDirectiveDirectiveSequenceCrawlingParameters paramNameparamInitialActionparamCrawlDirectiveparamDoDownload paramDoStoreMethodPOSTGETContainedTextFilter AttrFilterextractTagsContentfindAttributes noUrlFilter noAttrFilter noTextFilter unevaluated extractLinksextractLinksMatchingextractLinksWithAttributesextractLinksFilteringUrlAttrsextractLinksFilteringAllfindFirstLinkAfterfindAllUrlsEndingWithextractFirstFormsearchWebTemplatesearchWebTemplateAndProcessHitsCrawlingContextcrawlerdefaultContextstoringContextreadingContextexecuteCrawlChainexecuteActions crawlForUrl crawlChain crawlChainslogMsg delaySecondsReport reportMsg reportDetailsshowFullReport $fShowReport downloadTostoreDownloadAction TagContentTagSextractLinksFilteringgetSrc getTagAttrs isFormStart isFormClose isFormTag isFormStartOf tagAttributes extractFormextractFormParamfindExtraParamssplitOneOfRetainingNonEmptysearchWebActionfilterToUrlsContainingAllOffilterToUrlsContainingTextretainActionsContainingcrawl crawlAndStoreCrawlActionDescriberCrawler crawlInternal ajaxRequestbufferingFilenameDefaultCrawlingContextcrawlImplementation readFromFiles'$fCrawlingContextDefaultCrawlingContextlazyIOsequencefollowDirectivefollowDirectiveSequence wrapResults errReportokReportreportcrawlWasNoSuccess>>+makeAbsoluteLogicMappercombineAbsoluteUrlscombineAbsoluteUrl downloadSteplogAndReturnFirstOkputDetailsOnFailure