The AI Visibility Industry Might Be Built on Sand

There is a growing industry built on the promise of measuring something that may not be measurable in the way we've been told. AI visibility tracking has become a category almost overnight; venture-backed, rapidly expanding and increasingly expensive! Companies are spending an estimated $100 million per year on tools that promise to tell them where their brand "ranks" in AI search. The dashboards look familiar. The metrics sound authoritative. The vocabulary is borrowed directly from twenty years of SEO.

The problem is that AI does not work the way search engines work. And a landmark piece of research published in January 2026 has made that uncomfortably clear.

The SparkToro research

Rand Fishkin, co-founder of SparkToro, partnered with Patrick O'Donnell of Gumshoe.ai to conduct what may be the most comprehensive public study of AI recommendation consistency to date. Six hundred volunteers ran 12 identical prompts through ChatGPT, Claude and Google's AI a combined 2,961 times. Each response was normalised into an ordered list of brands, then compared for overlap, ordering and, of course, repetition.

The results were striking. There is less than a 1 in 100 chance that ChatGPT or Google's AI will return the same list of brands in any two responses to the same prompt. When it comes to ordering, the probability drops below 1 in 1,000. Claude performed marginally better at producing the same list, but worse at producing the same ordering.

Fishkin's conclusion was blunt: AI platforms are probability engines that generate a unique answer every single time. They do not consult a fixed table of rankings. Treating them as though they produce stable, measurable positions is nonsensical.

The uncomfortable question

This raises a challenge that the AI visibility industry has been reluctant to confront. If AI recommendations are inherently inconsistent, what exactly are these tracking tools measuring?

The familiar metrics; "you rank number three for this prompt in ChatGPT", "your position improved by two places this week"; are borrowed from a world where rankings were deterministic. Google's search results, for all their complexity, were broadly reproducible. The same query returned broadly the same results. That reproducibility made measurement meaningful.

AI search does not offer that reproducibility. The same query returns a different answer almost every time. A brand that appears first in one response may not appear at all in the next. Position tracking in this context is not just imprecise; it is misleading.

What the data actually tells us

However, I don't believe this research invalidates the entire category. Fishkin's own data reveals something he acknowledged but didn't fully unpack: whilst individual rankings are meaningless, visibility percentage is statistically meaningful.

Some brands appeared in nearly every response across hundreds of runs. Others barely appeared at all. City of Hope hospital, for instance, appeared in 97% of cancer care recommendation responses. That consistency, measured across volume rather than position, tells us something real about how AI models associate brands with categories.

The distinction matters enormously. The question worth asking is not "are you ranked number one in ChatGPT?" It is "does ChatGPT know you exist at all?" Those are fundamentally different problems, requiring fundamentally different strategies.

Why most businesses are invisible

The challenge for small and mid-sized businesses is not merely that AI recommendations are inconsistent. It is that many brands have no AI presence whatsoever; no visibility at any percentage, in any position, on any platform.

Research published alongside these findings offers a clue as to why. An estimated 85% of AI citations come from third-party sources rather than a brand's own website. AI models do not simply crawl your homepage and decide to recommend you. They synthesise information from review sites, industry publications, forums, structured data and authoritative third-party mentions. If other credible voices are not talking about you, these systems have no basis upon which to recommend you.

This explains a pattern I observe repeatedly. Businesses with strong Google rankings and well-optimised websites discover that AI platforms have no awareness of them. Their SEO investment, whilst valuable for traditional search, has not translated into the kind of distributed, third-party authority that AI models require.

Meanwhile, 37% of consumers now report starting their searches with AI tools rather than traditional search engines. That proportion is growing monthly. The shift is not theoretical; it is measurable and accelerating.

What actually works

If position tracking is meaningless and AI recommendations are generated probabilistically, what should businesses focus on?

The evidence points toward three priorities.

First, building genuine authority across the sources AI models trust. This means third-party reviews, industry publication mentions, community presence on platforms such as Reddit and Quora, and structured data that helps AI systems understand what your business does and for whom. This is harder and slower than optimising a landing page, but it is the work that creates durable AI visibility.

Second, measuring visibility as a frequency rather than a position. The meaningful metric is how often your brand appears across many runs of relevant prompts, not where it appears in any single response. This requires sufficient volume of monitoring to produce statistically meaningful data; something that single-point-in-time checks cannot provide.

Third, understanding the specific prompts and contexts where your brand is absent. The gap between "ChatGPT recommends your competitor 80% of the time" and "ChatGPT has never mentioned you" represents a spectrum of opportunity. Identifying the prompts where you are absent, and understanding which sources AI is citing instead, provides a clear and actionable starting point.

The work that matters

The AI visibility industry is not built entirely on sand. But a significant portion of what is being sold; position tracking, ranking dashboards, week-over-week position changes; has no statistical basis in how these systems actually operate.

What does have basis is frequency-based visibility measurement, combined with a clear understanding of the sources and signals that drive AI confidence in recommending a brand. That combination; knowing whether AI recognises you, understanding why or why not, and having a prioritised plan to address the gaps; is where the real value lies.

That conviction is what led us to build RivalScope. It tracks your brand's visibility across six AI platforms as a frequency metric, identifies the sources being cited when competitors are recommended and you are not, and provides a prioritised action plan in plain English. Not ranking positions. Not jargon. Clarity on what to do next.

If you are curious whether AI knows your brand exists, you can check your visibility for free. It takes roughly sixty seconds; and the answer, whether reassuring or sobering, is worth knowing.

Diana Young is the founder of RivalScope and writes on AI search, brand visibility, and digital marketing strategy.