Website Auditor

The AI Visibility Study

Search is becoming an answer, not just a list of links. This is an ongoing measurement of how often real small and mid-sized businesses are named when an AI assistant answers a buying-intent question about their category, and of what their own websites do or fail to do to earn that mention.

Built from 2,234 website audits across 1,048 domains and 91 detected sectors. The corpus runs to 2026-09-24, the date of its most recent audit. Free to cite and quote with attribution.

  • 89.8% of audited sites are never named in an AI answer
  • 4.0% of 30,340 assistant answers named the business asked about
  • 72.9% of answers naming a business with sources cited other websites, not its own

What is the AI Visibility Index?

The AI Visibility Index is an ongoing study of 1,048 business websites that measures three things separately: how often ChatGPT, Claude, Gemini and Perplexity name a business when asked a buying-intent question about its category and location, whose websites they cite when they answer, and which AI-readiness signals its own website publishes for an AI crawler.

How often do AI assistants name small businesses in their answers?

89.8% of 1,009 audited sites were never named in 30,340 answers from ChatGPT, Claude, Gemini and Perplexity to buying-intent questions about their own category and location. Only 4.0% of answers named the audited business, measured by the AI Visibility Index in the 2026-08-06 to 2026-09-24 window.

A customer asks an assistant for the best provider of a category in a place, and the assistant answers with a handful of names. This pillar asks whether the audited business is one of those names. Every audited business is queried against ChatGPT, Claude, Gemini and Perplexity, and each answer is matched against the audited business.

How machine-readable are small business websites to AI crawlers?

Across 1,048 sites audited for the AI Visibility Index, 83.0% publish robots.txt and 76.3% a sitemap, but only 53.7% carry schema.org data on the homepage and 44.1% LocalBusiness schema. 9.4% block at least one AI crawler in robots.txt, measured in the 2026-07-09 to 2026-09-24 window.

This pillar is what an assistant's crawler finds on arrival: whether robots.txt lets it in, whether the site publishes a sitemap, whether the homepage states the business's identity and location in machine-readable form. Every one of these signals is first-party, read by our own crawler directly from the site.

Presence is the outcome; readability is the input under a site owner's control. A site can be perfectly marked up and still go unnamed, which is why both are published side by side rather than combined into a single score. The signals come from the same free scan anyone can run on their own site.

What are the key findings of the AI Visibility Index?

89.8% of businesses audited for the AI Visibility Index are never named in an AI answer. Gemini names the audited business most often, in 4.4% of its answers. When an assistant names a business and shows its sources, 72.9% of those answers cite other websites rather than the business's own site.

  1. 89.8%

    of audited sites never surface in an AI answer

    Only 103 of 1,009 sites were named even once across 30,340 assistant answers to buying-intent questions.

  2. 72.9%

    of answers that named a business and showed sources cited other websites, not the business's own

    When an assistant named the audited business and showed its sources, it cited the business's own website in 182 of 672 answers. The five most cited domains across every answer, clutch.co, wikipedia.org, sec.gov, tripadvisor.com and reddit.com, drew 9,052 citations. In the answers to their own audits, the audited businesses' own websites drew 274 citations. When an assistant recommends a business and shows where it looked, what it shows is usually other websites.

  3. 4.4%

    Gemini has the widest reach in the panel

    Gemini named the audited business in 4.4% of its answers, ahead of every other assistant in the panel. All four were asked the same questions about the same businesses on the same days, so the gap is a difference between assistants rather than a difference between questions.

  4. 9.4%

    block at least one AI crawler in robots.txt

    98 of 1,048 sites disallow an AI user-agent in robots.txt. Their lists name crawlers that gather training data far more often than the search crawlers that let an assistant find and cite a page.

  5. 82.7%

    of sites that block an AI crawler block no search crawler this edition read

    81 of the 98 sites that block an AI crawler block only training-side crawlers. Those sites disallow no search crawler or user agent that this edition read, such as OAI-SearchBot or PerplexityBot, in their robots.txt. 57 of those 81 carry exactly the list Cloudflare's managed robots.txt wrote, so most of this split is a Cloudflare default rather than an owner's decision. That list also names Google-Extended, the Google token that governs grounding in Gemini as well as training data.

  6. 53.7%

    publish any schema.org structured data

    563 of 1,048 sites carry structured data on their homepage. The rest leave an assistant nothing to read but prose.

  7. 44.1%

    publish LocalBusiness schema

    The markup that states a business's name, address and category in machine-readable form is the least-adopted signal measured here, present on 462 sites. Most of this corpus is local businesses.

Which AI assistant has the widest reach for small businesses?

Gemini named the audited business in 4.4% of its answers, ahead of Perplexity 4.0%, ChatGPT 3.8% and Claude 3.6%, across 1,009 sites in the AI Visibility Index in the 2026-08-06 to 2026-09-24 window. All four assistants were asked the same questions about the same businesses on the same dates.

Share of answers naming the audited business 2026-08-06 to 2026-09-24 • 1,009 sites • 30,340 answers
  1. Gemini 4.4% 71 of 1,009 sites
  2. Perplexity 4.0% 58 of 1,009 sites
  3. ChatGPT 3.8% 64 of 1,009 sites
  4. Claude 3.6% 58 of 1,009 sites

Every assistant was asked the same questions about the same businesses on the same days, so the differences below are differences between assistants. Claude gave a readable answer to 7,240 of its 7,870 queries, the lowest share in the panel; the rest are excluded from its denominator rather than counted as absences.

AI crawlers blocked in robots.txt Share of 1,048 audited sites disallowing each user-agent
  1. Bytespider 8.6% 90 sites - training
  2. CCBot 8.2% 86 sites - training
  3. GPTBot 7.5% 79 sites - training
  4. Google-Extended 7.3% 76 sites - training
  5. ClaudeBot 7.2% 75 sites - training
  6. anthropic-ai 1.5% 16 sites - training
  7. PerplexityBot 1.3% 14 sites - retrieval
  8. OAI-SearchBot 1.1% 12 sites - retrieval
  9. ChatGPT-User 1.0% 10 sites - retrieval
  10. Perplexity-User 0.8% 8 sites - retrieval

Training crawlers such as GPTBot, ClaudeBot and CCBot collect pages that may be used to train a future model, which shapes what that model knows about a business. That influence arrives one model release at a time and carries no link. Search crawlers such as OAI-SearchBot and PerplexityBot build the index an assistant searches when it answers live, and user agents such as ChatGPT-User and Perplexity-User fetch a page on a person's behalf while an assistant composes its answer. Every assistant in this study was asked with web search available, so search crawlers decide whether a site can reach the answers it looks up in this study. Google-Extended is a control token rather than a crawler: it governs both Gemini training and grounding in the Gemini apps, so blocking it can also keep a site out of Gemini's live answers. User agents may not follow robots.txt at all, so a Disallow line addressed to them is a request rather than a control.

AI-readiness signals found on audited sites 2026-07-09 to 2026-09-24 • 1,048 sites • latest audit per domain
  1. Publishes robots.txt 83.0% 870 of 1,048
  2. Publishes sitemap.xml 76.3% 800 of 1,048
  3. Points robots.txt at sitemap 69.8% 731 of 1,048
  4. Has a meta description 75.2% 788 of 1,048
  5. Has Open Graph tags 71.5% 749 of 1,048
  6. Has any schema.org data 53.7% 563 of 1,048
  7. Has LocalBusiness schema 44.1% 462 of 1,048
  8. Blocks at least one AI crawler 9.4% 98 of 1,048

What is the methodology behind the AI Visibility Index?

The AI Visibility Index aggregates 2,234 free website audits across 1,048 domains, using the most recent audit per domain. Each business is queried against ChatGPT, Claude, Gemini and Perplexity with category-and-location questions, each answer's cited sources are matched against the business's own domain, and each site is read by Website Auditor's own crawler for robots.txt, sitemap and schema.org signals.

What is the source of the AI Visibility Index data?

The AI Visibility Index draws on 2,234 stored audits across 1,048 distinct domains in 91 detected sectors, run through Website Auditor's free scanner in the 2026-04-03 to 2026-09-24 window. Every domain was first audited by someone who chose it, and audits of the operator's own properties are excluded from every figure.

Later audits include scheduled re-runs of the same domains, so a site's latest audit reflects the current questions. The aggregate itself is committed as a SQL function and re-run for every edition, and any change to it is recorded in the changelog, so one edition is a comparison against the last rather than a fresh set of judgement calls.

How does the AI Visibility Index calculate each percentage?

Every AI Visibility Index percentage divides two integer counts, using one audit per domain. A site counts as present when at least one assistant named it in at least one answer. Refusals, unreadable answers and provider failures leave the denominator rather than counting as absences, so 1,009 of 1,047 audited sites enter the presence figure.

  • One domain, one vote. Both pillars use the most recent audit per domain. Counting audits instead would give a site scanned nine times nine times the influence over every share.
  • Answer presence covers 2026-08-06 to 2026-09-24: 1,009 sites and 30,340 assistant answers. A site counts as present if it was named in at least one answer from at least one assistant. Of the 1,047 sites audited in that window, 38 produced no answer that named any business and sit outside the denominator rather than counting as absences.
  • Machine readability covers 2026-07-09 to 2026-09-24 across 1,048 sites, a longer window and a larger sample, because these signals are read by our own crawler and depend on no third-party provider.
  • Unreadable answers are excluded, not counted as absences. A provider timeout, a refusal and a list our parser could not read are facts about the provider or about the parser, not evidence about the business. The assistant with the smallest readable share is named above, so a smaller denominator is something a reader can see rather than an inference.
  • Every percentage is a derivation. The published dataset holds integer counts only, and every share on this page is computed from two of them at render time. A headline therefore cannot drift away from its own data.

What are the limitations of the AI Visibility Index?

The AI Visibility Index corpus is self-selected, not a random sample: every site belongs to an owner who chose to run a free audit, which skews toward small and mid-sized businesses. Presence is more likely under-counted than over-counted because name matching is fuzzy, and the figures describe only the 2026-08-06 to 2026-09-24 window.

  • The corpus is self-selected, not a random sample of the web: every site in it is one whose owner chose to run a free audit, which skews toward small and mid-sized businesses actively working on their web presence. Treat the shares as descriptive of that population, not of all websites.
  • The cross-assistant panel admits no audit before 2026-07-23 because earlier audits fanned a single Perplexity answer across the ChatGPT, Claude and Gemini cards. Those earlier rows are real answers, but all of them are Perplexity's, so comparing assistants over them would measure one provider against its own output.
  • Claude gave a readable answer to 7,240 of its 7,870 panel queries, the lowest share of any assistant. The other 630 were refusals, lists our parser could not read, or provider failures (2 of them), and they sit outside its denominator rather than counting as absences.
  • Appearance is measured by matching the audited business against the businesses named in an assistant's answer. Name matching is fuzzy, and a business identified only by its domain, roughly one in five, is the hardest case. Presence is therefore more likely to be an under-count than an over-count.
  • Assistants are non-deterministic, and their indexes are in constant motion. These figures describe the answers given inside the stated window; the same queries run after 2026-09-24 will produce different answers.
  • Crawl signals are read from three places only: the homepage, robots.txt and sitemap.xml. A site carrying structured data on its inner pages but not on its homepage is counted here as a site with no structured data.
  • Crawler blocking is read from one source only, the site's robots.txt. A site can leave robots.txt open and still refuse an AI crawler at its server or its CDN, answering that crawler with an error while a browser gets the page, so the blocking shares here are more likely an under-count than an over-count. Claude-SearchBot and Claude-User, the search and user agents Anthropic runs beside ClaudeBot, joined the audit on 2026-09-25, after this edition's last audit, so neither appears in its crawler figures.
  • Crawler figures come from each site's latest audit between 2026-07-09 and 2026-09-24, so a site's robots.txt is the one it served on the day of that audit. On 2026-09-15 Cloudflare retired its managed robots.txt, which had written Disallow lines for GPTBot, ClaudeBot, Google-Extended, CCBot and Bytespider into each customer's robots.txt. Most sites here that block GPTBot carry exactly that list of five crawlers, and most of this edition's window predates the change, so the GPTBot and ClaudeBot counts can describe robots.txt files as they stood before that date rather than today. A re-read on 2026-09-25 found about half of the sites then carrying the list with it gone from robots.txt while Cloudflare's edge still refused GPTBot and ClaudeBot and served a browser, a pattern rare among Cloudflare sites that never carried the list. For those sites the refusal survives outside robots.txt. Part of the gap between training crawlers and retrieval agents is therefore a Cloudflare default rather than an owner's decision.
  • The structured-data figures in this edition are not comparable with the 2026-08-05 edition. Until 2026-08-02 the detector read only the top level of each JSON-LD block, so it missed the @graph container that Yoast and most of the WordPress ecosystem use to wrap their markup, and that edition under-counted both structured data and LocalBusiness schema. Audits in this edition are read by the corrected detector, so part of any change in these two signals since then is a correction of the measurement rather than a change in the sites.
  • Presence rates in this edition are not comparable with earlier editions. The denominator now counts answers a business name could be read out of, so a refusal or a list our parser could not read leaves it alongside a provider failure. Earlier editions counted both as absences.
  • Citation figures count only answers that showed their sources: 672 of the 1,199 answers that named a business. A site counts as cited when a source's domain is the audited domain or sits under that domain, and only in the answers to that business's own audit. Gemini links through its own redirect, so for Gemini the cited domain is the one it prints beside each link. The share of answers citing the business's own website ranges by assistant from 11.0% on Claude to 33.8% on Gemini.

Citation and reuse

Figures are free to reproduce with attribution and a link to this page. For the underlying methodology, or a cut of the data for a specific sector, write to support@website-auditor.io with your question.

What is the latest edition of the AI Visibility Index?

The latest edition of the AI Visibility Index is dated 2026-09-25 and covers 1,048 domains and 2,234 audits. Each edition updates the figures on this page and adds a dated changelog row instead of moving to a new URL, so a citation keeps resolving to current data.

  1. Fifth edition, four days after the fourth edition, published to add a new measurement rather than for new data. The Index now records whose pages the assistants cite when they answer, and that finding leads the page. It also counts sites per group of crawler, so the crawler finding is now a share of the sites that block any AI crawler. The panel grew from 983 sites to 1,009 sites and now runs to 2026-09-24, and the crawl-signal corpus grew from 1,012 sites to 1,048 sites. A slightly larger share of sites was named in an answer and a slightly smaller share blocks an AI crawler. Claude-SearchBot and Claude-User joined the crawler list on 2026-09-25, too late for any audit in this edition.

  2. Fourth edition, and the first past one thousand audited sites. The panel grew from 339 sites to 983 sites and now runs to 2026-09-21, and the crawl-signal corpus grew from 581 sites to 1,012 sites. The presence denominator changed with this edition: it now counts answers a business name could be read out of, so refusals and lists the parser could not read leave it alongside provider failures, and presence rates are not comparable with earlier editions. Structured-data figures are now read by the detector corrected on 2026-08-02, so they are not comparable with earlier editions either, for that separate reason. A smaller share of sites blocks an AI crawler. Gemini records the most appearances and now also the widest reach by distinct sites.

  3. Third edition, five days after the second edition. The panel grew from 193 sites to 339 sites and now runs to 2026-08-05, and the crawl-signal corpus grew from 436 sites to 581 sites. A slightly smaller share of sites was named in an answer. A larger share now blocks at least one AI crawler, a reversal of the direction reported by the last edition. Gemini still records the most appearances; the widest reach by distinct sites is now Perplexity. Edition ids carry a day from here on, because one month can hold more than one edition.

  4. Second edition. The panel grew from 175 sites to 193 sites and now runs to 2026-08-02, and the crawl-signal corpus grew from 418 sites to 436 sites. Every headline share moved by less than a point: a slightly smaller share of sites named in an answer, and a slightly smaller share blocking an AI crawler. Gemini keeps the panel's widest reach.

  5. First edition. Cross-assistant presence measured over 2026-07-23 to 2026-07-31, the first period in which ChatGPT, Claude, Gemini and Perplexity were each queried through their own provider. Crawl signals measured across 418 sites audited since 2026-05-16, at one audit per domain.

Where does your site sit in this data?

The Index is built from the same free audit anyone can run on their own site. It checks the AI-readiness signals measured above and queries all four assistants for your business by name, with no signup and a result in about a minute.

Run a free AI visibility audit