A set of options that you can use to specify what type of content is crawled, what URLs to crawl, and how deep and when to crawl.