A buyer asks ChatGPT "what's the best invoicing tool for a small agency?" and gets a short answer naming three or four products. If yours is not one of them, you never got the chance to compete for that buyer.
Generative engine optimisation (GEO) is the work of getting AI assistants to name you in answers like that. This guide explains what GEO is, how ChatGPT, Gemini, Perplexity and Claude find their sources (using only what each company says publicly), how to check where you stand, and which fixes are worth your time.
You can do most of it yourself, for free. Some popular GEO advice is a waste of time, and we cover that too.
What generative engine optimisation is
The term comes from a research paper, "GEO: Generative Engine Optimization" by Pranjal Aggarwal and five co-authors, first posted to arXiv in November 2023 and accepted at KDD 2024. The authors call search systems that gather several sources and summarise them into one answer with a language model "generative engines". GEO is their name for helping content creators become more visible in those answers.
They built a benchmark of 10,000 queries (GEO-bench) and tested nine ways of rewriting a web page. Adding quotations, adding statistics and citing sources did best, and they report that GEO "can boost visibility by up to 40%". Keyword stuffing did slightly worse than leaving the page alone, and the best method varied by subject. These are lab results from the researchers' own test engine and from Perplexity, not a promise about today's assistants.
Google takes a plainer view. Its guide to generative AI features in Search calls AEO (answer engine optimisation) and GEO industry terms, and says "from Google Search's perspective, optimizing for generative AI search is optimizing for the search experience, and thus still SEO."
In practice the groundwork is the same as SEO. What changes is what counts as a win and how you measure it.
GEO and SEO compared
| SEO | GEO | |
|---|---|---|
| Where you appear | A ranked list of links | Inside a written answer, sometimes with a source link |
| What a win looks like | A high position and a click | Being named, described correctly and cited |
| What the reader sees | Your title and snippet | The assistant's summary of you, next to others |
| How you measure it | Rankings, clicks and impressions in Search Console | Asking buyer questions yourself and logging the answers |
| What it depends on | Your pages, links and technical health | The same, plus AI crawler access and how consistently the web describes you |
How AI assistants find and cite sources
An assistant can know about you in two ways: from its training data, which stays fixed until the next model, or by searching the web while it answers. Answers with source links come from the second route, and that is the part you can change this month.
Each company documents its crawlers and how to control them in robots.txt, the file at your site's root that tells bots what they may fetch.
ChatGPT (OpenAI)
OpenAI says ChatGPT may search the web automatically when a question would benefit from current information. It often rewrites the question into targeted queries and may send them to third-party search providers. The page lists Microsoft's privacy statement among them, so make sure Bing can index your site, not just Google.
OpenAI's crawler page lists OAI-SearchBot for search, GPTBot for training and ChatGPT-User for pages a user's request calls for. Sites that block OAI-SearchBot "will not be shown in ChatGPT search answers, though can still appear as navigational links." The settings are independent, so you can allow search and block training.
Google AI Overviews, AI Mode and Gemini
Google says its AI features in Search rely on its core ranking systems and Search index, and may use "query fan-out": several related searches across subtopics to build one answer. To appear as a supporting link, a page must be indexed and eligible to show with a snippet, and Google says there are no additional technical requirements. Your site must also be included under Settings → Search generative AI in Search Console, which is the default.
Googlebot does the crawling. Google-Extended is a robots.txt token, not a crawler. It controls whether your content may train Gemini models and ground answers in Gemini Apps and Vertex AI. Google says it does not affect inclusion or ranking in Google Search, where AI Overviews and AI Mode live, but blocking it can keep your pages out of grounded answers in the Gemini app.
Perplexity
Perplexity documents PerplexityBot, which is "designed to surface and link websites in search results on Perplexity" and not used for foundation model training. Perplexity-User fetches pages when a person asks a question and generally ignores robots.txt, because a person started the request.
Claude (Anthropic)
Anthropic says Claude searches the web when a question benefits from current information, with citations in its answers. Its crawler page lists Claude-SearchBot for search, Claude-User for pages fetched when someone asks Claude a question, and ClaudeBot for training. Anthropic says all three follow robots.txt.
The crawlers at a glance
| robots.txt token | Company | Used for | If you block it |
|---|---|---|---|
| OAI-SearchBot | OpenAI | ChatGPT search results | Not shown in ChatGPT search answers |
| GPTBot | OpenAI | Model training | Content marked as not for training |
| ChatGPT-User | OpenAI | Pages fetched for a user | robots.txt "may not apply" |
| Googlebot | Search index, including AI Overviews and AI Mode | Google can't crawl your pages for Search | |
| Google-Extended | Gemini training and grounding in Gemini Apps | No effect on Google Search | |
| PerplexityBot | Perplexity | Perplexity search results | Perplexity recommends allowing it to appear in results |
| Perplexity-User | Perplexity | Pages fetched for a user | Generally ignores robots.txt |
| Claude-SearchBot | Anthropic | Claude search results | "May reduce your site's visibility and accuracy" in search results |
| Claude-User | Anthropic | Pages fetched for a user | Claude can't fetch your pages for users |
| ClaudeBot | Anthropic | Model training | Future content excluded from training |
Check whether you are named today
Start by finding out where you stand. It takes an afternoon.
- Write 10 to 15 buyer questions. Use a buyer's words, not your product name. Mix category ("best CRM for a five-person sales team"), problem ("how do I stop leads going cold"), comparison ("Product A vs Product B") and alternatives ("alternatives to Product A").
- Ask each assistant with web search on. Use a signed-out or fresh session where you can: OpenAI says ChatGPT may use your approximate location and saved memories when it rewrites a search.
- Log every answer: named or not, position, whether the description was right, who was named instead, which sources were cited.
- Open the cited sources. They show which review sites, forum threads and comparison articles the assistant relies on for your topic. These are your targets.
- Repeat every week or two. The same question can get a different answer tomorrow, so look for patterns, not single runs.
- Add Google's numbers. Search Console's Generative AI performance report shows how often links to your site appeared in AI Overviews and AI Mode.
A simple log:
| Buyer question | ChatGPT | Gemini | Perplexity | Claude | Named instead | Sources cited |
|---|---|---|---|---|---|---|
| Best CRM for a five-person sales team | Not named | 3rd, accurate | Not named | 2nd, old pricing | Product A, Product B | A review site, a Reddit thread, Product A's blog |
| Alternatives to Product A | 1st, accurate | Not named | 4th, accurate | Not named | Product B, Product C | A comparison article, your pricing page |
Practical fixes
The first two remove blockers. The rest give assistants a reason to name you.
1. Let the search crawlers in
AI crawlers are easy to block by accident with a security plugin, a CDN bot setting or a copied rule. Check yoursite.com/robots.txt. This setup allows AI search and opts out of training:
# AI search and answer engines
User-agent: OAI-SearchBot
User-agent: PerplexityBot
User-agent: Claude-SearchBot
User-agent: Claude-User
Allow: /
Disallow: /admin/
# Model training (your choice)
User-agent: GPTBot
User-agent: ClaudeBot
Disallow: /
# Everyone else, including Googlebot and Bingbot
User-agent: *
Disallow: /admin/
Sitemap: https://www.example.com/sitemap.xml
Three things to watch:
- Under the robots.txt standard (RFC 9309), a crawler that finds a group naming it ignores the
User-agent: *group. Repeat your private-path rules in every named group, as above. - Training and search are separate: OpenAI says its settings are independent, and Anthropic lists ClaudeBot (training) apart from Claude-SearchBot. Think twice before blocking Google-Extended if Gemini matters to you.
- If your CDN or firewall returns errors to these bots, the Allow line does nothing. OpenAI and Perplexity publish their crawlers' IP ranges so you can tell real visits from fakes.
2. Make key pages indexable and readable
Remove any noindex or nosnippet from pages you want quoted, since Google needs them indexed and snippet-eligible. Google can process JavaScript as long as it is not blocked, but the OpenAI, Perplexity and Anthropic crawler pages say nothing about rendering it. Put key facts in the HTML your server sends, and check the page source. Our technical SEO audit checklist covers the rest.
3. Write clear, factual pages about yourself
Assistants answer with facts, so make yours easy to find: what the product does, who it is for, how pricing works, what it integrates with and what it does not do. Write sentences that make sense when quoted alone. "Product A is invoicing software for agencies with 2 to 50 people" beats "We help teams do more."
The GEO paper found statistics, quotations and cited sources helped. The honest version: back claims with numbers you can stand behind, and link to where they come from. Google asks for first-hand experience and a point of view of your own.
4. Publish honest comparison and "best X for Y" pages
Buyers ask comparison questions, so give assistants a page that answers them. "Product A vs Product B" or "Best invoicing software for agencies" works when it is fair and says when the other option is better. Write one page per question your buyers really ask. Google says creating content for every fan-out variation mainly to manipulate AI answers breaks its scaled content abuse policy.
5. Earn mentions on other sites
Check the sources column in your log. If assistants cite review sites, forums and comparison articles, that is where you need to be. Google says its AI features can show "what's being said about products and services across the web, including in blogs, videos, and forum discussions," but that seeking inauthentic mentions "isn't as helpful as it might seem."
Get listed in the directories and review sites your buyers use. Help people where they ask questions (see Reddit marketing without getting banned). Launch where builders look, such as Show HN. Ask partners to list you on their integration pages.
6. Add Organization structured data
Structured data labels facts on a page for machines. Google says Organization markup on your home or about page can help it "disambiguate your organization in search results." It also says structured data is not required for its AI features, so treat this as housekeeping, not a shortcut.
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "Organization",
"name": "Example Co",
"url": "https://www.example.com",
"logo": "https://www.example.com/logo.png",
"description": "Invoicing software for agencies with 2 to 50 people.",
"sameAs": [
"https://www.linkedin.com/company/example-co",
"https://x.com/exampleco"
]
}
</script>
7. Keep your facts consistent
If your pricing page, LinkedIn page, an old press release and a directory listing describe you four ways, an assistant may repeat any of them. Write one sentence that says what you do and for whom, and use it everywhere. Check prices and plan names first.
What not to do
- Keyword stuffing. It did slightly worse than nothing in the GEO paper, and Google treats it as spam.
- Mass-produced pages. A page for every phrasing of a question is scaled content abuse in Google's terms.
- Fake mentions and reviews. Google says inauthentic mentions don't help.
- Hidden text for AI. White-on-white instructions or text moved off-screen count as hidden text abuse under Google's spam policies.
- Blocking search bots to stop training. Use the separate training tokens.
- Paying for guaranteed placement. Google says "no third-party tool has access to our internal ranking or AI systems."
- Expecting llms.txt to do the work. Google says Search ignores it. Our llms.txt guide explains what it is for.
How the BeforeQuery AI Marketer helps
The GEO Agent does the checking described above for you. On the free plan you get a weekly snapshot of 3 buyer questions. On paid plans it asks up to 15 buyer questions every day and writes 2 fixes per check. For each answer it records whether you are named, your position, the sentiment, which competitors are named and which sources are cited. The GEO tab shows whether ChatGPT, Gemini, Perplexity and Claude name you, and who they name instead.
The SEO Agent audits your homepage (once on the free plan, every day on paid plans), including AI search readiness: whether your robots.txt allows OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot and Bingbot, and whether the page can be read without JavaScript. The Coding Agent turns fixes such as robots.txt rules, Organization schema and an llms.txt file into a GitHub pull request. It never merges; you review and merge.
Nothing is published without your approval. To start, sign up free and enter your website.
Frequently asked questions
Is GEO different from SEO?
Mostly in what you measure. Google says optimising for its AI features is still SEO, because they use its core ranking systems. The groundwork is the same: crawlable, indexable pages with clear, useful content. GEO adds checking whether assistants name you correctly, letting AI search crawlers in, and paying closer attention to what other sites say about you.
Can I keep my content out of model training without leaving AI search?
Yes. OpenAI controls GPTBot (training) and OAI-SearchBot (search) separately, Anthropic separates ClaudeBot (training) from Claude-SearchBot (search), and Perplexity says PerplexityBot is not used for foundation model training. Google's Google-Extended token covers Gemini training and grounding in Gemini Apps without affecting Google Search, so blocking it may reduce how often the Gemini app uses your pages.
How long does GEO take to work?
No AI company publishes a timeline. OpenAI and Perplexity say robots.txt changes take up to about 24 hours to apply, and Google generally caches robots.txt for up to 24 hours. New pages still have to be crawled and indexed before they can be cited, and mentions elsewhere build slowly, so judge the trend over a couple of months.
Does structured data help AI assistants name me?
Google says structured data is not required for its generative AI features and there is no special schema markup for them. It does say Organization markup can help it tell your organisation apart from others in search results, and it still earns rich results. Add it as good practice, but expect clear pages and real mentions elsewhere to matter more.
Does an llms.txt file help with GEO?
Not in Google Search: Google says Search ignores llms.txt files and that they neither help nor harm rankings. The crawler pages from OpenAI, Perplexity and Anthropic do not mention it either. It is a short Markdown summary of your site meant for AI agents and used most for software documentation, so it is cheap to add but should not be your main GEO work.
Sources
- GEO: Generative Engine Optimization (Aggarwal et al., arXiv 2311.09735, KDD 2024)
- Google Search Central: Optimizing your website for generative AI features on Google Search
- Google Search Central: AI features and your website
- Google Search Central: Google's common crawlers (Googlebot, Google-Extended)
- Google Search Console Help: Search generative AI setting
- Google Search Console Help: Generative AI performance report
- Google Search Central: How Google interprets the robots.txt specification
- Google Search Central: Organization structured data
- Google Search Central: Spam policies
- OpenAI: Overview of OpenAI crawlers
- OpenAI Help Center: Searching the web with ChatGPT
- Perplexity: Perplexity crawlers
- Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler?
- Anthropic: Enable and use web search
- RFC 9309: Robots Exclusion Protocol
