Google has updated the documentation for the Google-Extended user agent, allowing publishers to control the use of their data for training purposes by Google Gemini and Vertex.
Updates Made
Google-Extended Product Token Description: The description was revised to enhance specificity and clarity based on publisher feedback.
Googlebot-News User Agent Description: The previous description incorrectly indicated that crawling preferences would affect the News tab on Google. This has been corrected.
Previous vs. Updated Documentation
Previous Documentation: It stated that Google-Extended is a standalone product token for managing site contributions to improve Gemini Apps and Vertex AI generative APIs, clarifying that disallowed pages would not be used for grounding.
Updated Version: The new documentation provides a clearer explanation of the user agent's purpose and the implications of blocking it. It specifies that the Google-Extended token allows publishers to manage whether their content can be used for training future Gemini models and for grounding in Gemini Apps and Vertex AI.
