You finally know who's reading.
Most "AI SEO" tools guess. WhoCrawls logs the actual hit. Per-bot counts, last-seen timestamps, the specific pages crawlers cared about.
How analytics workDrop one line of code. See exactly which AI bots crawl your site, on which pages, when. The free llms.txt generator is included. The dashboard is what you'll actually pay for.
No signup for the generator. No tracking. We log bot user-agents and nothing else.
Paste your domain. We'll crawl it (respecting your robots.txt) and build a clean llms.txt in about 30 seconds.
No email. No watermark on the file. We respect your robots.txt and crawl at 1 req/sec with a 30-page cap.
Most "AI SEO" tools guess. WhoCrawls logs the actual hit. Per-bot counts, last-seen timestamps, the specific pages crawlers cared about.
How analytics workPaste your domain. We crawl it (respecting your robots.txt), summarise with Claude, output a file you can drop at /llms.txt. Free. No email. No watermark on the file itself.
Try the generatorEvery hit gets a green tick if the source IP is in the bot's published range. Spoofed user-agents get a question mark. You see which is which.
What we verifyOne line in your <head>, or one line in your robots.txt. Both work. The snippet is 38 bytes. It doesn't load any third-party JS.
<link rel="ai-content" href="https://yourid.whocrawls.com/llms.txt">GPTBot, ClaudeBot, PerplexityBot, Applebot-Extended, Google-Extended, and seven others get logged the moment they touch the snippet or fetch your hosted llms.txt. Hits arrive in your dashboard within 60 seconds.
Per-bot hit counts. Most-crawled pages. Verified vs unverified IPs. A weekly email if you want one. No login walls between you and the data once you've paid.
The llms.txt generator. Watermarked share card. No account.
Generate freeGenerator + 30 days rolling dashboard history on 3 domains.
Buy Lite £15Unlimited history. Hosted endpoint. Weekly digest. 10 domains. 7-day trial.
Start 7-day trial| Feature | Free | Lite | Pro |
|---|---|---|---|
| llms.txt generator | Yes | Yes | Yes |
| Watermarked share card | Yes | No watermark | No watermark |
| Hosted endpoint | No | No | Yes |
| Dashboard | No | 30 days rolling | Unlimited history |
| Verified-IP green ticks | No | Yes | Yes |
| Weekly digest email | No | No | Yes |
| Install-issue surfacing | No | Yes | Yes |
| Domains | 1 | 3 | 10 |
| 14-day money back | n/a | Yes (UK Consumer Rights) | First month |
The free tier is the friendliest version. The paid tiers are where we stay alive. You'll know within a week whether the dashboard is worth £9/mo to you. If it isn't, the £15 Lite still buys you the analytics for 30 days. Cancel any time on Pro and your data stays accessible for 14 days.
Two paths. (a) You add a one-line snippet to your site, and any AI crawler that loads the page pings our edge endpoint. (b) You host your llms.txt at our subdomain (yourid.whocrawls.com/llms.txt), and any bot that fetches it gets logged server-side. Path (b) catches crawlers that skip JavaScript.
Two reasons. First, if GPTBot never visits, ChatGPT can't reference your docs. Second, if PerplexityBot stops crawling a key page, that's a signal something broke (a redirect, a 404, a robots.txt change). You want to know.
It's a markdown file at /llms.txt that tells AI crawlers what your site is about and which pages matter. Think sitemap.xml but for language models. You don't strictly need one. But if you have docs you want cited in AI answers, having a clean llms.txt helps.
No. Zero. We log bot user-agents. That's it. No IP addresses for humans, no cookies, no fingerprinting, no analytics SDK. Our privacy policy is unusually short for that reason.
The snippet is a 38-byte <link> tag. It does nothing on page load. It's a reference for bots, not a script. There's no measurable impact.
We check the source IP against the bot's published IP range. Verified hits get a green tick. Spoofed hits get a question mark. You see both, you can decide what counts.
Because some people don't want subscriptions and we don't want to lose them. £15 buys you the dashboard and 30 days of rolling history. If you keep getting value at day 31, upgrade to Pro for £9/mo.
UK Consumer Rights Act, 14 days no-questions. If you bought Lite and changed your mind before day 14, email us and we refund. If you bought Pro, your first month gets a refund the same way. After that, just cancel and we don't charge another period.
GPTBot (OpenAI), ChatGPT-User (OpenAI), ClaudeBot (Anthropic), Claude-Web (Anthropic), PerplexityBot (Perplexity), Applebot-Extended (Apple), Google-Extended (Google), Amazonbot (Amazon), Meta-ExternalAgent (Meta), Bytespider (ByteDance), Cohere-AI (Cohere), DiffBot. The list lives at whocrawls.com/bots and updates when bot providers ship new agents.
No. We support the standard (llmstxt.org) and our generator outputs to spec. But WhoCrawls is independent. We're not a standards body. We're a dashboard.