Google-Extended
Google-Extended explained
Google-Extended controls Gemini training. It does not control Google Search.
Google documents Google-Extended as a robots.txt token that lets a site opt out of being used to improve Gemini models. It is not Googlebot. Disallowing Google-Extended does not remove pages from Google Search, and allowing it does not change ranking.
The token is used in robots.txt like any other agent. Google's crawler overview lists it separately from Googlebot and from other special-case crawlers.
If a managed robots.txt, including some Cloudflare templates, blocks every AI token at once, check whether Google-Extended was included by accident.
What to check
- Search robots.txt for the Google-Extended group.
- Leave the Googlebot group unchanged if Search should still crawl the site.
- Read Google's common crawlers list before copying a long block list.
Fix prompt
Example prompt
Add a Google-Extended group in robots.txt if this host should opt out of Gemini training. Do not disallow Googlebot as part of that change. Do not call an indexing API.
Check a URL · Crawlers · Glossary
Questions
- Does Google-Extended affect AI Overviews?
- Google's own documentation is the source for what the token covers. It is documented as a control for Gemini model improvement, separate from Googlebot crawling for Search. Check that page if you need the current boundary.