• SpiderSettings


Namespace: Spider.Plugin

Assembly: Spider.Plugin

Contains setting values that determine how Spider.Plugin is configured.


ProcessStaticFiles Determines whether static files are ignored when determining whether a connection is allowed to connect to the resource.
StaticFileExtensions Delimited list of file extensions to ignore
BotTrapRoute Provides an opportunity to add a non existant route to sitemap which is dissalowed, if a bot goes to the route data is logged about the none behaving bot who is abusing sitemap. The value must be a valid root, i.e. /api/v19.x/ that is not in use elsewhere, this route is added to the sitemap and any visits to the route are notified.

In This Document