---
title: "PerplexityBot Explained: User Agent, IP Ranges, and the Stealth-Crawling Controversy | Kitbase Blog"
description: "PerplexityBot is Perplexity's search crawler. Its exact user-agent string, how Perplexity-User differs, published IP ranges, and the Cloudflare stealth-crawling dispute."
canonical: https://kitbase.dev/blog/perplexitybot-explained/
---

**PerplexityBot** is the crawler Perplexity uses to index the web for its answer engine — the bot that decides which pages Perplexity can surface and cite. Unlike OpenAI's [GPTBot](/blog/gptbot-explained/), Perplexity states that PerplexityBot's crawls are **not** used to train foundation models; its job is search indexing and citation. It identifies itself, respects robots.txt, and publishes IP ranges.

Perplexity is also the AI vendor with the most public controversy attached to its crawling — a 2025 Cloudflare investigation accused it of using undeclared "stealth" crawlers to reach content that had blocked its declared bot. Because that dispute changes how you should think about verifying Perplexity traffic, this guide covers both the documented bots and the controversy, with sources.

## PerplexityBot at a glance

| | |
|---|---|
| Operator | Perplexity |
| Purpose | Indexing content for **Perplexity search and citations** (not training) |
| robots.txt token | `PerplexityBot` |
| Current version | PerplexityBot/1.0 |
| Respects robots.txt | Yes (declared bot) |
| Executes JavaScript | No — fetches HTML |
| Published IP ranges | [perplexity.com/perplexitybot.json](https://www.perplexity.com/perplexitybot.json) |
| Official docs | [docs.perplexity.ai/guides/bots](https://docs.perplexity.ai/guides/bots) |

The user-agent string PerplexityBot sends:

```
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
```

Because Perplexity's crawler feeds its *search* index rather than a training corpus, being crawlable matters here in a very direct way: pages PerplexityBot can't read are pages Perplexity can't cite. If you care about [how Perplexity chooses citations](/blog/how-perplexity-chooses-citations/), allowing this bot is table stakes.

## PerplexityBot vs. Perplexity-User

Perplexity documents **two** agents, and the difference between them is the crux of the controversy below:

| Bot | Purpose | robots.txt token | Respects robots.txt |
|---|---|---|---|
| **PerplexityBot** | Crawls and indexes pages so they can appear and be cited in Perplexity | `PerplexityBot` | Yes |
| **Perplexity-User** | Fetches a specific page **when a user's question requires it** | `Perplexity-User` | No — treated as user-initiated |

Perplexity-User sends its own user-agent:

```
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)
```

Perplexity's stated position is that Perplexity-User acts on behalf of a human who asked for that page, so it "generally ignores robots.txt" — the same logic OpenAI applies to `ChatGPT-User`. Each agent has its own published IP list: [perplexitybot.json](https://www.perplexity.com/perplexitybot.json) and [perplexity-user.json](https://www.perplexity.com/perplexity-user.json).

## The 2025 Cloudflare stealth-crawling controversy

On **August 4, 2025**, Cloudflare published an investigation titled ["Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives."](https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/) Here are the documented findings, kept factual:

- **What Cloudflare observed:** When sites explicitly blocked Perplexity's declared crawler (via robots.txt or a firewall rule), Cloudflare says it saw follow-up requests from an **undeclared** user-agent designed to look like an ordinary browser — specifically a generic Chrome-on-macOS string:

  ```
  Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36
  ```

- **Scale:** Cloudflare reported the behavior "across tens of thousands of domains and millions of requests per day," with the stealth traffic in the range of a few million requests daily, alongside the much larger volume from the declared agent.
- **Evasion tactics:** Cloudflare says Perplexity **rotated source IPs and switched ASNs** in response to blocks, making the undeclared traffic hard to attribute.
- **Cloudflare's response:** it **de-listed Perplexity as a verified bot** and added managed-rule heuristics to block the stealth crawling.
- **Perplexity's defense:** Perplexity pushed back publicly, arguing that user-driven fetching is categorically different from automated crawling, and attributing some of the flagged behavior to a third-party service it used. Several commentators [defended that distinction](https://techcrunch.com/2025/08/05/some-people-are-defending-perplexity-after-cloudflare-named-and-shamed-it/); others sided with Cloudflare.

The takeaway for your own logs is not to adjudicate the dispute — it's practical: **a user-agent string is a claim, not proof.** If a vendor's traffic can appear under a browser-like UA from rotating IPs, then matching only on the string `PerplexityBot` will both miss real Perplexity fetches and accept fakes. Attribution has to lean on verifiable signals — published IP ranges and, increasingly, cryptographic signatures — not the UA alone. This is the same problem behind [Cloudflare-style AI crawler blocking](/blog/cloudflare-ai-crawler-blocking/).

## Verifying real PerplexityBot

For the *declared* bot, verification is the standard procedure: match the source IP against Perplexity's published ranges.

```bash
curl -s https://www.perplexity.com/perplexitybot.json
```

A request claiming to be PerplexityBot from an IP outside those CIDR blocks is spoofed — or, in light of the controversy, potentially undeclared traffic you'll never confirm as Perplexity at all.

**Classifying a request that claims to be PerplexityBot**

```mermaid
flowchart TD
  A["Incoming request<br/>UA contains 'PerplexityBot'"] --> B{"Source IP inside<br/>perplexitybot.json ranges?"}
  B -->|"Yes"| V["Verified PerplexityBot"]
  B -->|"No"| S["Spoofed or undeclared —<br/>not verifiable as Perplexity"]
```

Checking one vendor by hand is easy. Doing it continuously — refetching every AI vendor's ranges, matching each hit, and flagging the mismatches — is what should be automated. [Kitbase's bot and crawler detection](https://docs.kitbase.dev/crawler-detection) classifies every forwarded request server-side and stores each crawler visit with a **verified or spoofed** verdict, using published IP ranges and cryptographic [Web Bot Auth](/blog/web-bot-auth-explained/) signatures where present. Human visitors are classified in memory and discarded; only bots are stored.

## robots.txt recipes

PerplexityBot honors robots.txt. Note that blocking it removes you from Perplexity's *search* index — usually the opposite of what a marketing site wants.

**Allow Perplexity to index and cite you** (recommended for most sites — the default if you say nothing):

```
User-agent: PerplexityBot
Allow: /
```

**Block Perplexity's search crawler:**

```
User-agent: PerplexityBot
Disallow: /
```

**Block both agents:**

```
User-agent: PerplexityBot
Disallow: /

User-agent: Perplexity-User
Disallow: /
```

Two caveats matter more here than with most crawlers:

- **Perplexity-User may not honor `Disallow`.** Perplexity treats it as user-initiated, so a firewall rule on the published `perplexity-user.json` ranges is the only hard control.
- **robots.txt is honor-system, and Perplexity's compliance was publicly disputed.** If your intent is a genuine hard block, monitor whether it's actually being respected rather than assuming it is.

## Should you block PerplexityBot?

For most brands, blocking PerplexityBot is a mistake: it's a *search and citation* crawler, not a training crawler, so blocking it removes you from a discovery surface without protecting any training-data interest. If your goal is specifically to keep content out of AI *training*, PerplexityBot is the wrong target — block training crawlers like GPTBot and ClaudeBot instead, and see [should you block AI crawlers](/blog/should-you-block-ai-crawlers/) for the full framework.

Either way, **measure before you decide.** Look at what Perplexity actually reads on your site, whether the traffic verifies, and whether your pages show up in Perplexity's answers — before touching robots.txt.

## What to watch once you're tracking it

- **Which pages PerplexityBot reads** — these are the pages eligible to be cited. Gaps here explain gaps in your Perplexity citations.
- **Verified vs. unverifiable traffic** — given the controversy, the share of "Perplexity" hits you *can't* confirm is itself a signal worth watching.
- **Crawl-to-citation** — the crawl is the input; the output is whether Perplexity's answers actually mention and cite you, which [AI Visibility](https://docs.kitbase.dev/ai-visibility) measures by querying Perplexity's API directly. Seeing both sides turns "are we visible on Perplexity?" from a guess into a funnel.

## FAQ

**Is PerplexityBot used to train AI models?**
No. Perplexity states PerplexityBot indexes content for search and citations, not for training foundation models. That makes it primarily a discoverability crawler — blocking it removes you from Perplexity's answers.

**What's the difference between PerplexityBot and Perplexity-User?**
PerplexityBot crawls and indexes pages proactively and respects robots.txt. Perplexity-User fetches a specific page in response to a user's question and, per Perplexity, generally ignores robots.txt because it's user-initiated.

**Did Perplexity really use stealth crawlers?**
Cloudflare reported on August 4, 2025 that Perplexity used an undeclared browser-like user-agent and rotated IPs and ASNs to reach sites that had blocked its declared crawler, and de-listed it as a verified bot. Perplexity disputed the framing, arguing user-driven fetches differ from crawling. Both positions are linked above; the practical lesson is to verify by IP and signature, not by user-agent string.

**How do I verify PerplexityBot traffic is real?**
Match the source IP against [perplexity.com/perplexitybot.json](https://www.perplexity.com/perplexitybot.json). Anything outside those ranges is not verifiable as PerplexityBot. Kitbase [does this automatically](https://docs.kitbase.dev/crawler-detection) and labels every visit verified or spoofed.

**Should I block PerplexityBot?**
Usually no — it's a citation crawler, so blocking it hurts your visibility in Perplexity without protecting training data. If AI training is your concern, block training crawlers like [GPTBot](/blog/gptbot-explained/) and ClaudeBot instead.

---

*Want to see whether Perplexity reads and cites your site — verified and per-page? [Start your free trial](https://app.kitbase.dev/signup/) — 7 days, no credit card required — and track crawls and citations side by side.*