OpenAI
GPTBot
Fetching public pages that may be used to train OpenAI models.
- Who runs it
- OpenAI
- robots.txt token
- GPTBot
- User-agent
- Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot
OpenAI publishes the current GPTBot user-agent on its bots page. Match robots.txt on the token GPTBot.
GPTBot is OpenAI's training crawler. It is not the crawler ChatGPT search uses. That one is OAI-SearchBot. A person asking ChatGPT to open a link is ChatGPT-User.
OpenAI documents robots.txt as the control for GPTBot. A group named GPTBot does not change Googlebot.
Allowing GPTBot does not submit your site to ChatGPT. Blocking it does not hide you from Google. It is one crawler's permission.
Example robots.txt rules
Examples for your file, using the documented token. They are not a copy of the operator's own robots.txt.
Allow
User-agent: GPTBot Allow: /
Block
User-agent: GPTBot Disallow: /
Related guides
Other crawlers
Questions
- If I block GPTBot, can ChatGPT search still cite me?
- Search citations depend on OAI-SearchBot, which has its own robots.txt token. Blocking GPTBot only addresses the training crawler.