úÎ5‡.ħL      !"#$%&'()*+,-./0123456789:;<=>?@ A B C D E F G H I J K  SafeLMLMLM SafeNOPQRSNOPQRNOPQRSSafeSafeNoneTUVWTWTUVWSafe BAdds a prefix to a relative crawl action to get an absolute one. |      Safe    NoneXYXYXYSafe   Safe ˙A crawl directive takes a content of a web page and produces crawl actions for links/forms to follow. The general idea is to specify a list of operations that in theory produces a dynamically collected tree of requests which leaves are either dead ends or end results.òAdditional, logical branching/combination of Directives is possible with: * Alternatives - evaluate both Directives in order. * Restart - evaluate completely new initial action & chain if the previous combo does not produce end results..access content to find absolute follow-up urls0as simple, but found relative urls are completed-as simple, but with access to complete result (wait additional seconds before executing!5if given directive yields no results use add. retries"6fallback to second argument if first yields no results#=the possibility to start a new chain (when using alternative)$1not crawling anything, just a blacklisting option%chaining of directives  !"#$%  !"#$%  !"#$%  !"#$%Safe(name of crawl run)starting point*7list of operations sequentially on all previous results+Kstore the content of a single result (the first) of the last operation step,Hstore the url of a single result (the first) of the last operation step&'()*+,&'()*+,&'()*+,&'()*+,SafeZ(name, content, attributes)2TContent of the tags up to the first child tag (as simplification) - with attributes.!-./Z01[23456789:;<\=]^>?_`abcdefg-./0123456789:;<=>?2389:;<7>=?-./41506-./Z01[23456789:;<\=]^>?_`abcdefg Safe@Ahijk@A@A@AhijkNonel=Processes one step of a crawl chain: does the actual loading.mtUsed for preparation of integration tests: additionally stores the crawl result using the given file name strategy.nolmpnolmnolmp NoneqtMake a unique name for a crawl action - prefix is used to specify the target folder including a specific test prefix rstBCDEqFuvBCDEFBCDEFrstBCDEqFuvNonewLazily evaluate each action in the sequence from left to right, and collect the results. PS: also playing around with an additional concat before returning Gxyzw{|}~€‚  !"#$%G‚ Gxyzw{|}~€‚ NoneJXReturns only the first result of a completely matching branch of the crawling directive.KiReturns all possible results of the craling directive - meant to be used with lazyness in mind as needed.HIJKƒ„…GHIJKJKHIGHIJKƒ„…† !"#$%&'(()*+,-./01234566789:;<=>?@ABCDEFGHIJKLMN O P Q R S T UV W X Y Z [ \ ] ] ^ _ ` abcdefghijklmnopqrstu v w x yz{|}~  € €  ‚ ƒ„…†‡ˆ‰Š‹ŒŽ  ‘ ’“crawl_1Y5YW60vimoDjrTxEmhAjPNetwork.CrawlChain.StoringNetwork.URI.UtilNetwork.CrawlChain.CrawlAction'Network.CrawlChain.DirectiveChainResultNetwork.CrawlChain.CrawlResult!Network.CrawlChain.CrawlDirective%Network.CrawlChain.CrawlingParameters"Text.HTML.CrawlChain.HtmlFiltering!Network.CrawlChain.BasicTemplates"Network.CrawlChain.CrawlingContextNetwork.CrawlChainNetwork.CrawlChain.UtilNetwork.CrawlChain.ReportNetwork.Http.ClientFacadeNetwork.CrawlChain.DownloadingNetwork.CrawlChain.CrawlingNetwork.CrawlChain.CrawlChains buildCurlCmdbuildAndCreateTargetDirtoURI CrawlAction GetRequest PostRequest PostParamsPostType UndefinedPostFormPostAJAXcrawlUrl addUrlPrefixDirectiveChainResult resultHistory lastResultshowResultPathextractFirstResultCrawlingResultStatus CrawlingOkCrawlingRedirectCrawlingFailed CrawlResultcrawlingActioncrawlingContentcrawlingResultStatusCrawlDirectiveSimpleDirectiveRelativeDirectiveFollowUpDirectiveDelayDirectiveRetryDirectiveAlternativeDirectiveRestartChainDirectiveGuardDirectiveDirectiveSequenceCrawlingParameters paramNameparamInitialActionparamCrawlDirectiveparamDoDownload paramDoStoreMethodPOSTGETContainedTextFilter AttrFilterextractTagsContentfindAttributes noUrlFilter noAttrFilter noTextFilter unevaluated extractLinksextractLinksMatchingextractLinksWithAttributesextractLinksFilteringUrlAttrsextractLinksFilteringAllfindFirstLinkAfterfindAllUrlsEndingWithextractFirstFormsearchWebTemplatesearchWebTemplateAndProcessHitsCrawlingContextcrawlerdefaultContextstoringContextreadingContextexecuteCrawlChainexecuteActions crawlForUrl crawlChain crawlChainslogMsg delaySecondsReport reportMsg reportDetailsshowFullReport $fShowReport getRequestparseURLopenConnection ajaxRequest downloadTostoreDownloadAction TagContentTagSextractLinksFilteringgetSrc getTagAttrs isFormStart isFormClose isFormTag isFormStartOf tagAttributes extractFormextractFormParamfindExtraParamssplitOneOfRetainingNonEmptysearchWebActionfilterToUrlsContainingAllOffilterToUrlsContainingTextretainActionsContainingcrawl crawlAndStoreCrawlActionDescriberCrawler crawlInternalbufferingFilenameDefaultCrawlingContextcrawlImplementation readFromFiles'$fCrawlingContextDefaultCrawlingContextlazyIOsequencefollowDirectivefollowDirectiveSequence wrapResults errReportokReportreportcrawlWasNoSuccess>>+makeAbsoluteLogicMappercombineAbsoluteUrlscombineAbsoluteUrl downloadSteplogAndReturnFirstOkputDetailsOnFailure