Google has introduced two new additions to its Googlebot web crawler family: GoogleOther-Image and GoogleOther-Video. These crawlers are optimized versions of GoogleOther, which was launched in April 2023 for internal use by Google teams to crawl the public web and free up resources for the main Googlebot crawlers.
- GoogleOther-Image is designed for fetching publicly accessible image URLs. Its user agent tokens are GoogleOther-Image and GoogleOther, with the full user agent string being GoogleOther-Image/1.0.
- GoogleOther-Video is optimized for fetching publicly accessible video URLs. Its user agent tokens are GoogleOther-Video and GoogleOther, with the full user agent string being GoogleOther-Video/1.0.
Google launched these new crawlers to better support the crawling of binary data for research and development purposes.
The Googlebot crawler family includes:
- Googlebot, the main crawler for Google's search products, which always respects robots.txt rules.
- Special-case crawlers, performing specific functions, which may or may not respect robots.txt rules.
- User-triggered fetchers, tools and product functions triggered by the end-user.
Google also provides IP address ranges and reverse DNS mask for each type of crawler. When observing crawling activities and bot activity on your website and in your log files, do not be alarmed by the new GoogleOther crawlers, as they are genuine Googlebots.
