Skip to main content
Win Answer

Methodology. Reviewed 9 November 2026

What can be measured, and what cannot

The demo uses handmade sample data. Our own website audit below is a separate, real technical check. Future AI-answer measurements must include their method and limitations.

On this page
  1. Sample versus measured
  2. Our own site: a documented technical audit
  3. How the demo data was made
  4. What can be measured
  5. What cannot be measured
  6. Answers vary
  7. App or API: which route made the number
  8. Four things that get reported as one
  9. Standards we would hold ourselves to
  10. What we do not know yet

Sample versus measured

Illustrative demo

  • Demo answers, names and numbers are written by hand.
  • The concept preview does not query assistants or scan websites.
  • Samples do not represent customer results.
  • Our own technical checks are documented separately below.

What real measurement must include

  • Runs stored with date, route and settings.
  • Raw answers you can read.
  • A run count beside every result.
  • Ranges instead of a single score.

Our own site: a documented technical audit

On 8 November 2026 we audited winanswer.com after removing a site-wide indexing block. This is our own site, not a customer case study.

The live audit checked 140 localized pages: 112 public sitemap URLs and 28 intentionally excluded utility or demo pages. It reported zero check failures across canonicals, language alternates, indexing directives, structured data and internal links.

Google’s live test said /en/services was available to Google and could be indexed; indexing was requested. This does not confirm inclusion in search results.

Download the dated URL checks (CSV)

How the demo data was made

Everything in the concept preview, the sample answer on the home page and the panels on the product page was written by hand by the people building this, for demonstration.

  • No system was queried. No assistant, no API, no scraper, no search engine.
  • No real brands. Competitors are labeled A, B and C, so a name cannot collide with a real company.
  • Percentages and counts were chosen to show how a screen reads. They do not come from a distribution and say nothing about any industry.
  • Answers read like answers, but we composed them. No model returned them.
  • Photographs are stock images used as atmosphere. They do not show customers, clients or results.

What can be measured

Whether a brand is named in one sampled answerRead the answer.High for that run. It says little about the next.
How often a brand is named across repeated runsCount over many runs of the same question.Moderate, and improves with more runs. Report a range.
Which sources an answer showsRecord the citations shown.Moderate. Not every assistant shows sources, and a shown source is not proof it shaped the answer.
Whether stated facts match your pagesCompare the claim with the page.High, when the fact is checkable.
Change over timeRepeat the same questions after a change.Moderate, and only if the difference is larger than normal variation.

What cannot be measured

Demand
How often a question is asked. Providers do not share prompt volumes with outsiders, so any volume figure is an estimate of something hidden.
Private answers
What a logged-in person with a long history is told. Memory and location can change the answer, and a sample cannot reproduce it.
The reason
Why a system chose one brand. Google states that no third-party tool has access to its internal ranking or AI systems.
Exact rank
Order shifts between runs, which is why we report frequency.
Cause
Whether your fix produced the change. Some changes line up with an improvement and some do not.
Everyone
A set of runs is a sample of what an assistant can say, not a census of what buyers saw.

Answers vary

A study by SparkToro with the tracking vendor Gumshoe, reported in January 2026, ran 2,961 prompts across ChatGPT, Claude and Google’s AI results. For brand recommendation questions run repeatedly, the same list of brands appeared in under 1% of runs, and the same list in the same order in under 0.1%. How often a brand appeared at all was steadier than where it appeared.

The caveats matter. The study was not peer reviewed, one of its authors was working with a company that sells tracking, and the authors themselves called for larger follow-up work. Treat it as evidence for caution.

Illustrative example
One question, asked ten times

  • Competitor A

    Named in 8 of 10 runs

  • Competitor B

    Named in 6 of 10 runs

  • Your brand

    Named in 3 of 10 runs

Written by hand for illustration; it measures nothing. Competitor A appears in most runs, your brand in three. A single run would have told you almost nothing, and the order of names changes as well.

Illustrative example
Step through one question, sampled six times

A quiet boutique hotel near the old town

Run 1 of 6

  1. 1Competitor A
  2. 2Competitor B
  3. 3Competitor C

Your brand is not in this answer.

Your brand named in 0 of 1 runs so far

Written by hand for illustration. The same question gives a different list each time, and the order changes as well.

What we would do about it

  • Run every question many times and show the count.
  • Report how often a brand is named, as a range, never a rank.
  • Mark any change smaller than normal variation as noise.

App or API: which route made the number

There are two main ways to collect an answer. The consumer app is what a buyer sees. A developer interface is easier to run in volume, and may use a different model version, different tools such as web search, and different settings. The two can disagree.

Surfer, a company that sells AI search tracking, published a comparison in August 2026 of 1,000 prompts and 13,779 answers across several products. It reported brand lists overlapping only about 15.5% to 23.8% between app and API results as returned. In its data the ChatGPT app cited 12.1 sources per answer against 3.1 through the API. Each prompt was run once, so part of the gap may be ordinary variation, and the vendor sells the scraped approach it favors. Read it as vendor data.

Consumer app, sampled by hand or by toolClosest to what a buyer sees.Slow and hard to scale. Automated collection may be restricted by the provider’s terms, which we would check first.
Developer interfaceEasy to repeat at volume.May not match what buyers see.

Four things that get reported as one

Four different things often get reported as a single visibility number. We would keep them apart.

Named
Your brand is mentioned in the answer.
Recommended
Your brand is named with a favorable reason, or placed as a suggested choice.
Cited
Your pages are linked as a source.
Described correctly
The facts in the answer match the facts on your pages.
Illustrative example
An answer that cites you and recommends a competitor
  • The answer links to your guide as a source for how to choose.
  • It then recommends two other products by name.
  • Cited: yes. Recommended: no. A tool that counts links would call this a win.

Standards we would hold ourselves to

  • Show the run count and the date beside every result.
  • Keep raw answers readable next to the summaries.
  • Label the assistant, route, market and language of each figure.
  • Show ranges. Publish no single visibility score without publishing how it is made.
  • Say when a difference is within normal variation.
  • Keep a public log of corrections to our own claims.
  • Never promise a ranking, a mention or a result.

What we do not know yet

  • How many runs per question are enough. Researchers have raised this as open.
  • Whether answers differ between languages in ways that matter. We think they may, and have not tested it.
  • Whether changes you make move what assistants other than Google’s say.
  • How far changes can be linked to traffic or inquiries without overclaiming.

These are the things we hope to learn with design partners, and we would publish what we find, including when it is disappointing.

Sources and how we used them

Checked on 6 November 2026. Where a primary document exists, we link it. A figure from a company that sells into this market is labeled vendor data, since whoever sells the measuring has an interest in the result.

Keep reading

Think our method is missing something?

Tell us. A design-partner conversation is the best place to challenge how we would measure your market.

Talk to us about your buyers