See who's crawling your site
AI assistants and search engines read your content around the clock, and your normal analytics deliberately hide them. Kitbase makes them visible: which crawlers arrive, which pages they fetch, and whether they are who they claim to be.
Every request resolved to a named crawler
A hit isn't just a user-agent string. Kitbase resolves each one to the crawler and the vendor behind it, so you can tell who is reading your content and how much of it.
- AI crawlers named individually — GPTBot, ClaudeBot, PerplexityBot and hundreds more
- User-agent claims are verified against the crawler's published identity
- Per-path frequency shows what's being read and what's ignored
- Spot a new or unusually aggressive crawler the moment it appears
A user-agent string proves nothing
Anything can claim to be Googlebot — spoofing a header is trivial, and scrapers do it constantly to get past naive blocks. Kitbase checks each claim against the crawler's published identity, so a verified hit means the bot really is who it says.
- Every detected request is marked verified or spoofed
- Real crawlers are separated from things wearing their name
- Attribution resolves the crawler and the vendor behind it
Two features, opposite goals
Kitbase does two different things with bots. It helps to know which one you're looking at — both use the same pipeline to decide what counts as a bot, they just do opposite things with the answer.
| Bot & Crawler Analytics This page — opt-in | Bot filtering Built into web analytics — automatic | |
|---|---|---|
| Goal | Make bots visible — see which crawlers and AI bots visit your site | Keep bots out of your human analytics |
| Where it runs | Your server or edge, forwarding requests to Kitbase | The JS SDK in the browser, plus the server-side pipeline |
| What happens to bots | Stored as their own dataset with attribution — name, vendor, verified or spoofed | Silently discarded; they never reach your dashboards or billing counts |
| Setup | Opt-in — install a forwarder for your platform | Automatic, no configuration |
Pick your platform and paste
Detection runs at your server or edge, forwarding requests to Kitbase's ingest endpoint with your secret API key. Humans are ignored at the endpoint, so it's safe to forward everything.
The first time you open the Crawlers page, a wizard walks you through creating the key, installing the forwarder, and live-verifying that the first crawler arrives.
Crawlers visiting you, not answers about you
Bot analytics tells you which AI crawlers fetch your pages. AI Visibility tells you whether those engines actually mention your brand when someone asks. They're complementary — one measures the input, the other the output.
- Bot analytics — which crawlers arrive, how often, and on which paths
- AI Visibility — whether ChatGPT, Gemini and Perplexity name you in answers
- Together they connect what gets read to what gets said
Find out who's reading your content
Install a forwarder and watch the crawlers arrive — named, attributed, and verified against spoofing.
No credit card required