Open topic with navigation

# MaxSpiderThreads

The `MaxSpiderThreads` parameter specifies the maximum number of threads used by a single fetch task to perform page requests. During spidering, the connector builds a list of URLs to retrieve. When a thread becomes available, it is assigned a URL to process and downloads the page. The thread becomes available for the next URL when the download and processing are complete.

If the server’s response time is longer than the time spent by the connector processing the page (that is, if threads are frequently waiting for pages to be returned), it may be advisable to increase the value of this parameter.

You should also consider the load on the HTTP server - more threads will result in requests being sent more frequently.

Type: | Integer
Default: | 1
Required: | No
Configuration Section: | TaskName or Default
Example: | `MaxSpiderThreads=2`
See Also: |

© 2016 Hewlett Packard Enterprise Development LP | Send documentation feedback to HPE Close We welcome your comments! To open the configured email client on this computer, open an email window. Otherwise, copy the information below to a web mail client, and send this email to AutonomyTPFeedback@hpe.com. Product: Topic Title: Help Topic: Feedback:

_HP_HTML5_bannerTitle.htm