A third party looking through your website for content, such as pages, images, and other files.
These are typically performed by search engines like Google and Bing for indexing purposes, and other applications like SEO and performance tools, for testing and validating page content.