Chrome extension · interactive walkthrough

What GPTBot actually receives

Googlebot renders your JavaScript. GPTBot, PerplexityBot and ClaudeBot don't. This is the real UI, wired to a real audit of a client-rendered blog post. Pick a crawler: the page redraws, and the tool tells you why that crawler can or can't read it.

24lowest score
13/17blocks unseen
3/4headings unseen
2bots turned away

The page being audited

https://northwind.example/blog/ai-answer-engines
AI Crawlability LensRed = invisible to this bot
© 2026 Northwind Analytics · Privacy · Terms

The extension popup

420 px — actual size

AI Crawlability Lens

What GPTBot, PerplexityBot and ClaudeBot actually see.

https://northwind.example/blog/ai-answer-engines

Per-bot results

Botrobots.txtFetchScore

Visibility Score = share of your rendered content blocks present in that bot's raw HTML, headings weighted double. Red under 40, amber 40–75, green above 75.

Inspect a bot

Framework and recommendation

Next.js (App Router) (low confidence)

Ensure this content renders in a Server Component, not a Client Component ('use client') — move data fetching to the server.

A 'use client' boundary high in the tree pushes everything below it to the browser. Fetch in an async Server Component and pass the resulting data down as props, keeping 'use client' for leaf-level interactivity only.

Signals: App Router RSC payload (self.__next_f / /_next/static/chunks/app/)

Recent checks

  • Why AI answer engines can't see most of the web Today, 07:19 · lowest 24 (Googlebot) · 1 robots-blocked
    24
  • Getting started — Northwind Today, 07:17 · lowest 24 (Googlebot) · 1 robots-blocked
    24
  • Pricing — Northwind Today, 07:17 · lowest 24 (Googlebot) · 1 robots-blocked
    24
100% local. No account, no server, no telemetry. The only requests made are to this page and its robots.txt, and only when you click Check.

Why the numbers read the way they do

first failure winsIt's a gate chain, not a score

A request passes five gates: allowed by robots.txt → server responds → response is HTML → content is in the source → crawler runs JavaScript. Only the first failure is the cause; gates after it read skipped, not failed, because they were never reached.

A page can be robots-blocked and fully client-rendered. Reporting both equally sends you to rewrite a frontend when the fix was one line of robots.txt.

same page, two verdictsGate 5 changes everything

Googlebot and ClaudeBot get identical scores of 24 here. Googlebot reads "at the mercy of rendering" — a warning, since it will probably catch up. ClaudeBot reads "reaches it, cannot read it" — critical, because it never runs JavaScript and never will.

One measurement, two verdicts, because the crawlers differ. That distinction is the entire product.

no score, not zeroGPTBot shows n/a

GPTBot is disallowed in robots.txt. It never requests the page, so a Visibility Score would be a number about content it will never fetch. The tool refuses to print one.

blocked ≠ low scorePerplexityBot shows 403

The server turned that user-agent away. That's a firewall problem, not a rendering problem — conflating the two is exactly what this tool exists to stop.

separate tokenGoogle-Extended isn't Googlebot

It's not a crawler and has no user-agent. It governs whether Google may use your content for Gemini and Vertex AI grounding. Allowing Googlebot does not allow it — and almost nobody knows.

headings ×2Why 4 of 17 scores 24

Answer engines lean on headings for passage selection, so headings count double. Visible weight 5, total weight 21 — 24, not the 24% a flat count would give.

no invented findings"Specific to your User-Agent"

The tool only claims a 403 is user-agent-specific when the plain-UA baseline got a 200. If the baseline was refused too, the site is blocking everyone — and saying otherwise would be a fabricated finding.

forbidden headerHow the user-agent gets set

fetch() silently drops a User-Agent header. A scoped declarativeNetRequest rule sets it instead, locked to the extension's own requests, one bot at a time.

never mutatesThe overlay draws over

Highlights live in a Shadow DOM layer positioned over the page. Verified byte-identical before, during and after — measuring the page must not change it.