On 30 October 2026 Google said it plans to honour a shared AI training signal in robots.txt, based on work from the IETF AIPREF working group, and aims to support it together with other AI providers in 2027. One robots.txt group will tell participating AI companies whether your pages may be used to train generative models. For a business that wants to be cited by ChatGPT, Gemini and AI Overviews, the change brings one practical risk: Google plans to read a blanket training opt-out with no Google-Extended group as a Google-Extended block, and it ties Gemini citations to that setting.
What Google announced on 30 October 2026
Google Search Central published "Supporting a shared AI training control for websites", signed by John Mueller. The post makes four points, quoted from Google's announcement:
- Websites will use a shared policy token in robots.txt to state AI training preferences across participating AI providers in one step.
- When a site opts out of training with that token and does not name Google-Extended, Google plans to treat it "as if the publisher had disallowed Google-Extended".
- Company-specific controls such as Google-Extended, offered since 2023, keep working alongside the shared signal.
- "Websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps."
The mechanism is described in the individual Internet-Draft draft-kelley-aipref-training-preference-signal-00, dated 30 October 2026. It adds no new directive: preferences are ordinary robots.txt groups keyed by the token __usage_ai-training. The draft's own example:
User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/This says "no AI training" for the whole site except /press-kit/. Three details from the draft matter for site owners. A User-Agent: * group never counts as a training preference. The token is a usage preference and leaves crawling rules untouched. A file with no __usage_ai-training line expresses no preference at all. The draft is a work in progress and may change before 2027.
Three switches, three different results
Google now documents separate controls for training, for AI answers in Search and for Search itself. Its "Control what you share with Google Search" page, updated on 21 October 2026, lists the Search generative AI control next to noindex, robots meta, data-nosnippet and X-Robots-Tag.
| Control | Where you set it | What it governs | Status |
|---|---|---|---|
Shared training token __usage_ai-training | robots.txt | Training preference for participating AI providers; Google plans to read an opt-out as a Google-Extended block when Google-Extended is not named | Google aims to support it in 2027 |
| Google-Extended | robots.txt | Use of content for Gemini; sites that allow it are eligible for grounding clicks in Gemini Apps | Live since 2023 |
| Search generative AI control | Search Console → Settings → Search generative AI | Inclusion in AI Overviews, AI Mode and generative features in Discover; Google says it "doesn't affect AI training" | Rolled out to all websites worldwide as of 31 August 2026 |
| noindex | Robots meta tag or X-Robots-Tag | Removal of the page from Google Search entirely | Live |
Per Google's help page, the Search generative AI control defaults to Include, is not used as a ranking signal in other parts of Search, and an exclusion generally takes a few days to take effect.
What it means if you want AI assistants to cite you
For most service businesses, being quoted in AI answers brings enquiries, so the settings that keep content visible deserve a deliberate choice. A developer or plugin that adds one "no AI training" group could also remove the site from Gemini Apps grounding, because Google plans to treat that group as a Google-Extended block whenever Google-Extended is missing from the file. Google limits that reading to files where Google-Extended "is not explicitly specified", and the draft says specific expressions should take precedence over general ones. An explicit User-Agent: Google-Extended group is therefore the clearest way to state your choice for Google.
The reverse case also comes up. A firm that wants to keep its paid course materials out of training can close one path with the shared token and leave service pages, prices and FAQs open. The draft binds preferences to paths, so a site-wide decision is optional.
What to check before 2027
- Read your live robots.txt. List every AI user agent it names: Google-Extended, GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and others. Note which ones are blocked and who added the rule.
- Decide by purpose. Write down three answers: may models train on our pages, may AI answers in Search quote us, may Google Search index us. Each answer maps to a different control in the table above.
- Name Google-Extended explicitly if you want Gemini Apps citations, so your choice for Google stays stated even if a blanket opt-out is added later.
- Open Search Console → Settings → Search generative AI and confirm the property shows Include, unless you have a reason to leave AI Overviews and AI Mode.
- Keep commercial pages open. Service descriptions, prices, locations and FAQs are what an AI assistant needs in order to recommend a business. Close only paths you sell or keep private.
- Skip
User-agent: *for AI choices. Under the draft a wildcard group never expresses a training preference. - Diary a review for early 2027. The Internet-Draft can change before Google switches support on; recheck the final syntax then.
Marketing MIX, an Ottawa agency working since 2012, keeps its own robots.txt open to Google-Extended, GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot, because citations in AI answers are part of how clients find us. The same audit is the first step of our AI SEO agency work and of an AI-readable website rebuild. Businesses in the capital region can start with our AI agency in Ottawa at 208 Booth Street.
FAQ
What is the shared AI training control in robots.txt?
A robots.txt group keyed by the token __usage_ai-training that states whether pages may be used to train generative AI models. Google announced on 30 October 2026 that it plans to support it with other AI providers in 2027.
Does blocking AI training remove my site from AI Overviews?
Inclusion in AI Overviews and AI Mode is a separate switch: the Search generative AI control in Search Console. Google says that control does not affect AI training, and the training token governs training only.
What happens if I use the shared token and do not mention Google-Extended?
Google plans to treat a training opt-out as if Google-Extended were disallowed. Google says sites that allow Google-Extended are eligible for grounding clicks in Gemini Apps.
Do I need to change robots.txt now?
Google has not switched support on yet. Use the time to audit the file, name Google-Extended explicitly and confirm the Search Console setting, then recheck the final syntax in 2027.