Field notes/Research

Who AI names when your buyers ask

Four assistants, four different shortlists. What the published citation research shows about where those answers come from — and the robots.txt mistake that quietly keeps brands out of them.

The Comma team

Short answer

AI assistants name brands they can parse quickly and confirm somewhere else. The published citation analyses agree on two points: community and reference sites are cited far more often than brand sites, and what ranks on Google is largely not what gets cited. Ranking still helps. It is not the same job.

The answers rarely come from where you think

The instinct is that winning on Google wins the AI answer too. The published data keeps saying otherwise. Ahrefs found that only about 12% of the URLs cited by assistants such as ChatGPT, Gemini and Copilot rank in Google's top 10. Perplexity is the outlier, at roughly 29%.

Meanwhile, analyses running to tens of millions of citations put Reddit as the single most-cited domain across ChatGPT, Google AI Mode, Gemini and Perplexity. Not a brand site. A forum.

Both things are true at once: ranking still helps, and ranking is not the same job. The page that wins your keyword is frequently not the page that gets quoted.

Each assistant has a different diet

Ask four assistants the same buyer question and you will often get four different shortlists. That is not randomness. They draw on different sources and weight them differently.

  • Conductor's seven-month study found ChatGPT and ChatGPT Search are the only engines that surface Wikipedia, while Gemini cited YouTube across every intent it tested.
  • Meltwater's October 2026 report records Gemini cutting its Reddit citations by around 88% as it began citing fewer sources overall, and Perplexity capping visible sources at about 10, down from roughly 24, in early October.

These are vendor studies with different methods, sample windows and engine lists, so treat them as direction rather than gospel. The durable point survives the caveats: a brand can lead on one assistant and be absent from another, and the fix is usually different for each.

It also means the mix moves. A source that mattered in March can be discounted by October. Whatever you measure, measure it again.

The robots.txt mistake almost everyone makes

This is the one worth checking today, because it is binary and most published advice gets it wrong.

The standard tip is "let the AI crawlers in — allow GPTBot and ClaudeBot." Read the vendor documentation, though, and both of those are training crawlers. Neither has anything to do with whether you appear in an answer.

What the vendors actually document

  • OpenAI: GPTBot trains foundation models. OAI-SearchBot is what surfaces you in ChatGPT's search. ChatGPT-User is user-initiated, and OpenAI notes robots.txt may not apply to it. Their documentation says plainly to use OAI-SearchBot to manage search.
  • Anthropic: ClaudeBot is training. Claude-SearchBot is the one that indexes you for search. Claude-User fetches pages when a user's question needs it. Three separate tokens, three separate decisions.
  • Perplexity: PerplexityBot is search. Perplexity-User is user-initiated and generally ignores robots.txt. Perplexity documents no separate training crawler.
  • Google: Google-Extended is not a crawler at all. It is a usage signal covering Gemini training and grounding, and Google states that blocking it does not affect your inclusion or ranking in Google Search.

The practical upshot

You can allow every training crawler and still be invisible in answers, or refuse training entirely and keep full answer visibility. Training access and answer visibility are separate decisions, and most robots.txt files conflate them by accident.

Open your own robots.txt and look for the search tokens, not the training ones. It takes two minutes and it is the cheapest item on this list.

What seems to move the rest

Beyond crawler access, the pattern across the published analyses is consistent enough to act on, even where nobody has cleanly separated cause from correlation.

Answer the question outright, near the top

Assistants extract. A page that opens with a warm-up paragraph offers nothing to lift. A page whose first two sentences answer the question directly hands over a quotable unit. This is the cheapest change and usually the first to show up.

Publish a comparison that admits trade-offs

A comparison page that names competitors fairly, including where they are the better fit, reads as reference material. A feature grid that only flatters you reads as a brochure. Reference material is what gets cited.

Make your facts agree with each other

Pricing, category and positioning should match across your own site, your review-site profiles and your listings. Where sources disagree, a model has to pick, and it often picks neither.

Show up where buyers actually talk

Given how heavily community threads are cited, the threads discussing your category are part of your surface area whether you take part or not. Read them, and correct on your own pages whatever they get wrong about you.

What to do this quarter

No large programme required. In rough order of effort:

  1. Check robots.txt for OAI-SearchBot, Claude-SearchBot and PerplexityBot — the search tokens, not the training ones.
  2. Write down the 20 questions your buyers actually ask, in their words rather than your category's.
  3. Run all 20 on ChatGPT, Claude, Gemini and Perplexity. Record who is named, in what order, and which pages are cited.
  4. Rewrite your three most important pages so the first two sentences answer the question.
  5. Publish one honest comparison page.
  6. Find the community threads cited for your top questions, and note what they say about you.

Then run the same 20 questions again a month later. The comparison over time is the measurement. Any single snapshot is noise.

Where this comes from

Comma is new. This post is built on published citation research and the crawler documentation each vendor maintains — all of it linked above, so you can check it rather than take our word for it. It is not a proprietary study of our own.

We would rather say that plainly than imply a dataset we do not have. When we do have our own, we will publish it with the method attached, including the parts that did not work.

Make the list

Thirty minutes. We run your buyer questions on ChatGPT, Claude, Gemini and Perplexity, and show you where you stand.

Talk to sales