answers.delivery

How-to · 3 min read

How to see which AI crawlers read your website

Site owners who want to show up in ChatGPT or Perplexity answers usually have no idea whether those engines ever read their pages in the first place. Google Analytics shows people, not bots, so a crawler from OpenAI or Anthropic leaves no trace there. You might be blocking them by accident, or they might read your pricing page every hour and skip your docs entirely. You want a plain log of which AI bots came, what they read and why.

Published
17 Nov 2026
Delivery
No. 12
Topic
AI and agents
Evidence
9 claims quoted from usenotra.com

01
Why it's hard

Most analytics tools work through a JavaScript tag that runs in a browser, and AI crawlers generally fetch the raw page without running scripts, so they never register. Raw server logs do capture them, but they are long, noisy and full of user agents you have to look up one by one. Even when you spot a bot, the name alone rarely tells you whether it was training a model, refreshing a search index or fetching a page to answer a live question. And fake scrapers often borrow well known bot names, which makes the numbers look better than they are.

This part describes the usual way. It makes no claims about Notra, so nothing is quoted here.

02
How Notra does it

Notra logs AI bot visits on your server side instead of in the browser, so crawlers that skip JavaScript still show up. It matches each request against 66 known AI agents and tags the visit as a training crawl, a search index refresh, a live answer or a referral click. A pages view shows which URLs each engine keeps reading and which ones it has never opened. Setup means adding an ingest token to your middleware or edge config, and the free plan covers agent and web analytics up to its event allowance.

AI crawlers skip JavaScript, so tag-based analytics miss them. Notra logs each request on your server, so you see which pages each engine read.Quoted from usenotra.com
Notra tags each visit as a training crawl, a search index refresh, a live answer or a referral click.Quoted from usenotra.com
Add your ingest token to your middleware or edge config. It takes a few minutes and works on any stack.Quoted from usenotra.com

03
Step by step

  1. 1Sign up for the free plan and create a project for your site.
  2. 2Add your ingest token to your middleware or edge config, or install the SDK if you run Next.js, Nuxt, TanStack Start, Astro or SvelteKit.
  3. 3Wait for traffic, then open the bot visits log to see each request with its time, provider, path and purpose.
  4. 4Check the pages view for URLs that no engine has opened yet.
  5. 5Fix or link those pages, and watch whether live answer visits start to reach them.

04
Tips

Look at live answer visits separately from training crawls. A training crawl means a model may learn from the page later, while a live answer visit means an engine fetched it to reply to someone right now, which is much closer to a citation.

Hits from 66 known AI agents, sorted by what they were doing: training a model, building an index, reading a page to answer someone or a person clicking through.Quoted from usenotra.com

Pay most attention to the pages engines never open. A comparison or pricing page that no bot has read cannot be quoted in an answer, so it is the first place to add internal links or check your robots rules.

See which pages each engine keeps reading and which ones it has never opened.Quoted from usenotra.com

Do not count a crawl as a win. A bot reading your page is not the same as an engine recommending you, and Notra keeps those two apart on purpose.

We don't count a fetch as a citation.Quoted from usenotra.com

Be suspicious of a sudden spike from one famous bot name. Scrapers often spoof user agents, and matching against known crawlers helps keep a stray scraper from being counted as ChatGPT.

Notra matches each request to a known AI crawler and labels why it came, so a stray scraper never counts as ChatGPT.Quoted from usenotra.com

05
Questions

Can I track AI crawlers on the free plan?

Yes. The free plan covers agent and web analytics with a shared allowance of 1 million events a month.

Track AI agents and people visiting your site with one shared event allowance, and collect agent feedback.Quoted from usenotra.com

Which frameworks does the SDK support?

Next.js, Nuxt, TanStack Start, Astro and SvelteKit, with server-side capture.

One package for Next.js, Nuxt, TanStack Start, Astro and SvelteKit.Quoted from usenotra.com

Does a bot visit count as being cited?

No. A fetch is logged as a visit, not as a citation in an answer.

We don't count a fetch as a citation.Quoted from usenotra.com

More from Notra

All 2 answers