Assembly: Spider.PluginContains setting values that determine how Spider.Plugin is configured.
|ProcessStaticFiles||Determines whether static files are ignored when determining whether a connection is allowed to connect to the resource.|
|StaticFileExtensions||Delimited list of file extensions to ignore|
|BotTrapRoute||Provides an opportunity to add a non existant route to sitemap which is dissalowed, if a bot goes to the route data is logged about the none behaving bot who is abusing sitemap. The value must be a valid root, i.e. /api/v19.x/ that is not in use elsewhere, this route is added to the sitemap and any visits to the route are notified.|