AI model behavior
Why assistants mention or skip brands, and how training crawlers differ from retrieval crawlers.
Updated 2026-08-06
AI assistants do not behave like a single blue-link ranking algorithm. Mentions emerge from a mix of training memory, live retrieval, prompt context, and safety/policy filters. VisibleMax helps you improve the signals those systems can use — it cannot rewrite vendor models.
What assistants tend to cite
In practice, assistants are more likely to name a brand when public content makes the entity easy to identify and compare:
- Clear entities — consistent brand, product, and company names (not only metaphors or taglines).
- Comparisons — explicit category fit, alternatives, and differentiation.
- FAQs — direct answers to buyer questions in plain language.
- Structured facts — pricing posture, who it is for, integrations, and proof points.
- Machine-readable aids — schema, sitemaps, and
llms.txtwhere relevant (supporting signals, not magic).
Training crawlers vs retrieval crawlers
| Kind | Typical purpose | Visibility implication |
|---|---|---|
| Training / foundation crawlers | Collect content that may inform future model training | Affects long-horizon knowledge; blocking them does not remove you from live search-style answers overnight |
| Retrieval / search crawlers | Index or fetch pages for grounded, up-to-date answers | Often the more direct path to appearing in AI search and cited answers |
Examples of the split include OpenAI’s GPTBot vs OAI-SearchBot, Anthropic’s ClaudeBot vs Claude-SearchBot, and Perplexity’s PerplexityBot / Perplexity-User fetchers. Allowing retrieval bots you care about is usually the higher-leverage robots.txt decision for AI answers.
Why brands get skipped
- Ambiguous positioning (“AI platform for everything”) with no concrete offer.
- Thin public HTML or SPA shells that crawlers and retrievers cannot read.
- Stronger competitors with clearer comparison and review coverage.
- Prompt mismatch — the buyer question does not match how you describe yourself.
- Index lag or provider-side retrieval gaps unrelated to your on-page quality.
FAQ
If I allow GPTBot, will ChatGPT recommend me?
Not automatically. Training access and live recommendation are different. Prefer also allowing retrieval/search bots and publishing clear, citable pages.
Do probes equal what every user will see?
Probes are controlled API/assistant checks with fixed prompts. End-user chats vary by personalization, tools, region, and model version.
Related
Still stuck? Contact support with your project URL and what you already tried.