Every merchant who has wondered whether an AI assistant recommends their store has already run the test. You open a chat window, type best <your product> to buy online, read the answer, and draw a conclusion. It takes a minute and it costs nothing, and this is not an article telling you it was a waste of a minute.
It is an article about what that minute measured, because it is a different thing from what Cited measures, and the difference is the entire reason the tool exists. Every claim below about how Cited works was read out of its engine's source and checked against the deployed configuration this morning.
Which Question the Chat Window Answers
The engine's provider layer opens with a design constraint written above everything else, and it is the whole comparison in one sentence. The providers must be queried “in its WEB-GROUNDED mode (live retrieval / RAG), NOT the bare base model — otherwise we measure training memory instead of real AI search.”
That is the distinction your manual test does not control. A chat window may or may not go and search before it answers you. When it does not, the answer comes from whatever the model absorbed during training: a snapshot, months old, in which a store that launched last spring may simply not exist and a store that closed last year may still be recommended. When it does search, the answer comes from what it retrieved just now. Two answers, produced by two different mechanisms, and from the outside they look identical: a paragraph of confident prose with some brand names in it.
The difference, stated once
The manual test measures memory. The tool measures retrieval. Neither is wrong. If you want to know what the model already believes about your category, the chat window answers that. If you want to know whether a shopper who asks today, with search on, will hear your name, you need the answer to come from the grounded mode every time, on purpose, and you need to know that it did.
What a Hit Actually Is
The second thing the eyeball test cannot do is separate three outcomes that feel like one. Reading an answer, you register “we were in it” or “we were not”. The scoring code keeps three separate scores, and they answer different questions.
Mentioned means one of your brand terms — the store name, any alias you gave it, or your domain — appears in the answer text. Cited is stricter and separate: your own domain appears among the sources the provider attached to its answer. An assistant can name you without linking you, and it can link you in a footnote without naming you in the prose, and those are different results for a shopper. Prominence is where the first mention lands: an answer that opens with your store scores high, one that mentions you in the last clause scores low, and one that does not mention you at all scores zero. Competitors you list are checked the same way, by name, in the same text.
A merchant reading one answer conflates the first two and cannot see the third at all, because prominence is a position, not a feeling, and a position only means something across many answers.
Prompts Come From the Store, Not From You
The third difference is the question itself. You typed one query, the one that came to mind, in the phrasing that came to mind. Cited generates its prompt set from the store: eight buyer-intent templates (“best {x} to buy online”, “where to buy {x}”, “{x} worth the money”, and the rest) filled from your product types and collections, falling back to your product titles if you have neither. The app asks the engine for that set, and you edit and approve it before anything runs.
The point is not that generated prompts are cleverer than yours. It is that a single question you chose measures the answer to a question you chose, and a shopper does not ask your question. A set that covers the intents shoppers actually have, phrased several ways, is a measurement. One query is an anecdote, however carefully typed.
Which Assistants, and What Happens When One Cannot Answer
Honesty about scope, because a comparison that only lists strengths is a sales page. The engine declares five provider names. Its own status line says three are live — an OpenAI web-search mode, Perplexity, and Gemini with Google Search grounding — and two “remain stubbed until their adapters land”. I confirmed on the production box this morning that keys exist for exactly those three, that the default run uses exactly those three, and that mock mode is off. So when Cited reports a result, it is three assistants, each queried in its grounded mode, not five and not a guess.
A provider that fails is an error, not a result
The engine can also produce deterministic placeholder answers so the pipeline runs without keys during development. In production that switch is off, and in live mode a provider with no key, or one that times out or rejects the call, raises. The run does not stop: that provider is recorded by name in the run's error list and left out of the score entirely, and the report shows a “couldn't reach” line naming it. What it never does is fill the gap with placeholder text and score it. A blank you can see is worth more than a number you cannot trust, which is the same principle as a log line being a claim rather than evidence.
What the Manual Test Still Wins
Several things, and they are real.
- It is free and it is now. Sixty seconds, no install, no account. If the answer is a clear yes or a clear no in your category, you have learned something.
- You read the whole answer. A score is a summary. The actual paragraph, with its tone and its reasons, is something only a human reading it gets, and it is often where the useful hint is.
- You can follow up. A chat window lets you ask “why not us?” A measurement tool, by design, asks the same fixed questions every time so the numbers are comparable.
The trade is the same one every comparison on this shelf comes down to: the strengths and the limits come from the same place. The manual test is one question, one time, in whichever mode the assistant happened to use, read by a person who wants a particular answer. That makes it fast and rich and impossible to compare against last month.
Choosing Between Them
One question settles it
Do you want to know what the model remembers, or whether a shopper searching today will hear your name? The first is a chat window. The second needs the grounded mode on purpose, a prompt set wider than one query, three separate scores instead of one impression, and a run you can repeat next month against this one. That is what Cited is, and it is all it is.
What it is not is a lever. Cited measures. It does not make a store appear in an AI answer, and nothing in this article should be read as a promise that any change will move a result; the assistants decide what they say, and the tool's job is to tell you truthfully what they said. The measurement itself, what each score costs to produce and why the run size is capped, is walked through in inside Cited, and its place among the other six apps is in the other shelf. If you arrived here from a link with the wrong app name in it, yesterday's article explains why that happens; the listing itself is at apps.shopify.com/cited.