|
|
|
Description |
This module provides all the settable options in shpider.
|
|
Synopsis |
|
|
|
Documentation |
|
|
Setting this to True will forbid you to download and sendForm to any site which isn't on the domain shared by the url given in setStartPage.
|
|
|
Set the CurlTimeout option. Requests will TimeOut after this number of seconds.
|
|
|
Set the start page of your shpidering antics.
The start page must be an absolute URL, if not, this will raise an error.
|
|
|
Return the starting URL, as set by setStartPage
|
|
|
If onlyDownloadHtml is True, then during download, shpider will make a HEAD request to see if the content type is text/html or application/xhtml+xml, and only if it is, then it will make a GET request.
|
|
|
Set the given page as the currentPage.
|
|
|
Return the current page
|
|
|
When keepTrack is set, shpider will remember the pages which have been visited.
|
|
Produced by Haddock version 2.4.2 |