Perplexity
PerplexityBot
Fetching pages Perplexity uses for answers.
- Who runs it
- Perplexity
- robots.txt token
- PerplexityBot
- User-agent
- Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot
Perplexity publishes PerplexityBot and a separate Perplexity-User agent. Confirm the current strings in their bot guide.
PerplexityBot is the crawler Perplexity documents for collecting pages used in answers. Perplexity-User is the fetch a person requests. They are different tokens.
Perplexity documents robots.txt as the way to allow or block PerplexityBot. The page still needs its text in the HTML if you want an answer to have something to quote.
A block here does not block Google or OpenAI.
Example robots.txt rules
Examples for your file, using the documented token. They are not a copy of the operator's own robots.txt.
Allow
User-agent: PerplexityBot Allow: /
Block
User-agent: PerplexityBot Disallow: /
Related guides
Other crawlers
Questions
- What about Perplexity-User?
- Perplexity documents it as the user-triggered agent. Give it its own robots.txt group if you need a separate rule.