What the tracker counts, and what it doesn't

Watch · 3:02

• 0:07Two kinds of AI traffic
• 0:44Why the tag can't see crawlers
• 1:13How a visit gets its label
• 2:04How to see crawler visits
• 2:33Reading the tracker honestly

In shortThe AI bot tracker counts what a JavaScript tag on your pages can see: people who click through from an AI answer, and AI agents that load your page in a real browser. It does not see AI crawlers such as GPTBot, ClaudeBot or PerplexityBot, because they fetch your HTML without running scripts, so the tag never fires for them. To see crawler visits, read your server or CDN logs.

Two kinds of AI traffic

"AI traffic" means two different things, and they show up in different places.

• AI crawlers are programs that fetch your pages so an AI company can train a model or build a search index. They request your HTML, read the text and leave. Examples are GPTBot, ClaudeBot and PerplexityBot.
• AI-referred visitors are people. Someone asks ChatGPT or Perplexity a question, sees your page cited, and clicks the link. Their browser loads your page like any other visit.

There's a third, smaller group: AI agents that drive a real browser to complete a task for someone. Because they use a browser, they run your page's scripts.

Why the tag can't see crawlers

The tracker works through a small script you add to your pages. The script only runs when a browser executes JavaScript. Most AI crawlers download the raw HTML and never execute it. For them, the script is just text in the page, so nothing is ever sent to AI Rank Lab. This is how JavaScript analytics works in general, not a setting you can change.

How a visit gets its label

When the script runs, it decides what kind of visit it is, in this order:

1. User agent. If the browser's name contains a known AI name (for example ChatGPT, Claude, Perplexity or Copilot), the visit gets that label.
2. Referrer. Otherwise, if the visitor came from a known AI site, such as chatgpt.com or perplexity.ai, it's labelled with that engine. Most AI rows you'll see come from this rule.
3. Generic words. Otherwise, if the user agent contains short strings such as "bot", "crawler", "spider" or "ai", it's labelled AI Bot.
4. Everything else is a human visit, later split into Google, direct and other sources.

The tag also stores UTM parameters, but it doesn't use them for the label. A click from an AI app that sends no referrer, even with utm_source=chatgpt.com in the link, is counted as an ordinary human visit.

What shows up where

How to see crawler visits instead

If you want to know whether AI crawlers fetch your pages, you need logs from the machine that serves them. This is general advice; the exact steps depend on your host.

• Server access logs. Most hosts let you download them. Search for user agent names such as GPTBot, ClaudeBot, PerplexityBot or OAI-SearchBot to see which URLs were fetched and when.
• CDN logs and bot reports. If your site sits behind a CDN, its dashboard may group requests by bot or show AI crawler activity.
• Check before you trust a name. Anyone can fake a user agent. Some AI companies, OpenAI and Anthropic among them, publish the IP ranges their crawlers use, so compare against those where they exist.
• robots.txt decides access. Whether crawlers may fetch your site is set in robots.txt. If you block them there, an empty log is the expected result.

Reading the tracker honestly

• Call the AI numbers AI-referred visits, not crawls, even where a card says "crawls" or "indexing", or a progress report says "AI crawler visits".
• Trust rows with an AI referrer most. Treat AI Bot rows as unclassified.
• Expect some under-counting: AI apps that don't pass a referrer look like direct traffic.
• Use the tracker for trends and for which pages AI answers send people to. That's what it measures well.

AI Rank Lab covers the other side through brand visibility, which shows whether AI answers cite your pages in the first place. Together they show citations going in and visits coming out.