Track AI Bots on Vercel with Log Drains (Zero Code)

The fastest way to track AI crawlers on Vercel is a Log Drain — no middleware, no redeploy, no code. Vercel already logs every request that hits your site, including the ones from crawlers that never run JavaScript. A Log Drain streams those logs to an endpoint of your choice; point it at Kitbase and every GPTBot, ClaudeBot, PerplexityBot, and Googlebot visit shows up in your dashboard, classified and geolocated, minutes after you save the drain.

This matters because your existing analytics can’t see any of this. GPTBot and its peers fetch server-rendered HTML and leave — they never execute the tracking snippet that JavaScript analytics depends on, so tag-based tools report zero AI crawler traffic whether or not the bots are actually there. The request logs, on the other hand, record every hit. This guide shows you how to turn those logs into an AI-crawler dashboard on Vercel in about five minutes.

Why the log drain, not middleware

There are two ways to capture crawler requests on Vercel. You can add edge middleware that forwards each request as it happens, or you can use a Log Drain that ships Vercel’s own request logs after the fact. Both land the same data in Kitbase; the trade-off is code versus configuration.

If you’re on a Pro or Enterprise team, the Log Drain is the zero-friction path — you never touch your codebase, and it captures static asset and edge requests that middleware matchers often miss. (Log Drains are a Pro/Enterprise feature on Vercel; on the Hobby plan, use the middleware forwarder instead.)

How it works

A Vercel Log Drain POSTs batches of request logs to your endpoint as JSON or NDJSON. Kitbase reads each log’s proxy object — the request metadata Vercel captures at the edge — and pulls out the signals it needs to classify the actor:

• userAgent — the claimed identity (e.g. GPTBot/1.4)
• clientIp — the real client IP, used to verify the crawler against its vendor’s published ranges
• method, host, path — what was requested
• referer — where it came from

Kitbase classifies each request server-side. Human visitors are identified in memory and discarded — only bot and crawler requests are stored, so forwarding your entire request stream never turns into a privacy liability or a storage bill. For the crawler hits it keeps, the raw IP is used to derive geolocation (country, region, city) and then dropped, unless you’ve explicitly enabled IP logging for the project.

flowchart LR
A["Crawler hits<br/>your Vercel site"] --> B["Vercel request log<br/>(proxy object)"]
B -->|"Log Drain POST"| C["Kitbase ingest"]
C --> D{"Human or bot?"}
D -->|"Human"| E["Discarded"]
D -->|"Bot / crawler"| F["Stored + verified<br/>+ geolocated"]

Setup

The whole thing is a form in your Vercel settings. Follow the Vercel forwarder guide as the canonical reference — the steps below mirror it.

1. Go to Vercel → your Project or Team → Settings → Log Drains (or use the Vercel API).
2. Endpoint: set it to https://ingest.kitbase.dev/ingest/v1/vercel. Your secret API key (step 5) is what resolves the target Kitbase project — there’s nothing else to configure on the URL.
3. Sources: select Static and Edge. Add Function if you render pages on-demand and want crawler hits on those routes too.
4. Delivery format: JSON or NDJSON — Kitbase accepts both.
5. Custom header: add Authorization: Bearer sk_kitbase_<your_secret_api_key>. This authenticates the drain and resolves your project. Use the secret API key (starts with sk_kitbase_), not the browser SDK key — the SDK key is public and won’t work here.
6. Endpoint verification: Vercel runs a one-time ownership check when you create the drain. Kitbase answers it automatically by echoing the x-vercel-verify header, so there’s nothing to click.

Save the drain. That’s the entire integration — no git push, no build.

Confirming it works

A 200 OK response from the ingest endpoint carries a small JSON body:

{ "accepted": 3, "received": 214 }

received is how many request logs Kitbase parsed from the batch; accepted is how many of those were bots or crawlers it stored. A low accepted relative to received is expected and correct — most of your traffic is human, and humans are discarded by design. Non-request logs (build output, function console.log lines with no proxy object) are simply ignored. Within a few minutes the crawler detection dashboard starts filling with named bots.

What arrives in the dashboard

Once the drain is live, each AI crawler stops being a line in a log file and becomes a queryable dataset:

• Which bots visit, by name — GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Googlebot, and the rest, matched from the user-agent token rather than a brittle full-string match. The full list of AI crawlers explains what each one is for.
• Verified vs. spoofed — Kitbase checks each hit’s IP against the vendor’s published ranges (and validates a Web Bot Auth signature when the crawler sends one). A request claiming to be GPTBot from outside OpenAI’s ranges is flagged as an impostor, not counted as GPTBot.
• Which pages they read most — the paths crawlers favor are the pages the models learn about you from. Documentation, blog posts, and comparison pages usually dominate.
• Crawl trends over time — a sudden stop often means an accidental block (a WAF rule, a robots.txt change) you’d otherwise never notice; a spike can track model-training cycles.
• Geography — derived from the IP at ingest time, then the IP is dropped.

This is the crawl stage of the AI discovery funnel: are the engines reading your site at all? The next stages — whether those crawls turn into citations, and whether citations turn into clicks — are separate datasets that Kitbase’s AI Visibility and web analytics measure. Seeing crawler traffic on Vercel is where the funnel starts.

Vercel Log Drain vs. other hosts

The Log Drain pattern is Vercel’s version of a broader idea: let the platform’s own request stream feed your crawler analytics instead of instrumenting application code. Other hosts have equivalents worth knowing if you run a mixed stack:

• Cloudflare — a Cloudflare Worker forwards requests with ctx.waitUntil, which also pairs naturally with Cloudflare’s own crawler controls.
• AWS CloudFront — standard logs to Firehose, another zero-code path.
• Your own nginx — ship the access log as JSON, ideal for self-hosters and VPS setups.

All of them land in the same dashboard with the same verified/spoofed classification, so a multi-host site sees every crawler in one place.

FAQ

Do I need to redeploy my Vercel app to add crawler tracking? No. A Log Drain is pure configuration in your Vercel settings — no code, no build, no redeploy. Crawler data starts flowing within minutes of saving the drain.

Which Vercel plan do I need for Log Drains? Log Drains are available on Vercel’s Pro and Enterprise plans. On the Hobby plan, use the Next.js edge-middleware forwarder instead, which works on any plan and captures the same data.

Why can’t Google Analytics or Plausible see these bots? AI crawlers fetch HTML and never run JavaScript, so they never execute a tag-based analytics snippet. Those tools can only count visitors who run their script — which by definition excludes every non-JS crawler. Log-based capture sees them because the log records every request regardless of JavaScript.

Will forwarding all my request logs store data on every human visitor? No. Kitbase uses human requests only to classify them in memory, then discards them. Only bot and crawler requests are persisted, and even for those the raw IP is dropped after geolocation unless you enable IP logging.

Which sources should I select on the drain? Static and Edge cover most crawler traffic. Add Function if you render pages on-demand (SSR/ISR on-request) and want crawler hits on those routes captured too.

Want to see which AI crawlers read your Vercel site — verified, attributed, and per-page? Start your free trial — 7 days, no credit card required — and point a Log Drain at Kitbase in five minutes, zero code.