What Happens Between Your Question and the Citation: How AI Answer Engines Pick Sources

Otlume GEO series, part 2 of 10 · Level: Beginner · By Otlume's GEO optimization expert

In July 2025, Ahrefs published a study with a very comforting headline. 76% of AI Overview citations pull from the top 10.

I remember how neat that felt. If three quarters of what Google's AI cites already ranks on page one, then GEO for Google is mostly SEO done well. Rank, and the AI will follow.

Then, on March 2, 2026, Ahrefs published an update. Same question, bigger dataset, 863,000 keyword results pages and about 4 million AI Overview URLs. The new number was 38%.

From roughly three in four to roughly two in five, in eight months.

I'm not bringing this up to dunk on Ahrefs. They re-ran their own study and published the inconvenient answer, which is exactly what you want from a data source. I'm bringing it up because it's the best single illustration I know of the thing this article is about. You can't treat an AI answer engine as a fixed machine with fixed rules. It's a pipeline. The pipeline changes, and the rule of thumb you learned last year can quietly expire.

So let's look at the pipeline. Not the secret internals, nobody outside these companies has those. Just the shape, which is public and pretty stable.

In part 1 I said the unit of competition shrinks from a page to a passage. Here's why.

The original GEO paper (Aggarwal et al., KDD 2024) describes a generative engine that, at the time, looked a lot like Bing Chat. A user asks something. A model rewrites that question into several simpler search queries. Those queries go to a regular search engine, which returns ranked sources. Another step summarizes or extracts from those sources. Then a final model writes the answer, citing the sources it leaned on. In their own test setup they kept it simpler, fetching the top 5 Google results for each query and having a model write an answer from them.

That's retrieval plus generation, and almost every system you care about works some version of it when it's answering a question that needs fresh or specific information.

Google says so fairly openly. Its "AI features and your website" page explains that AI Overviews and AI Mode "may use a 'query fan-out' technique," issuing multiple related searches across subtopics and data sources to build a response. So when someone asks "is a heat pump worth it in a cold climate," the system may also be quietly searching things like heat pump efficiency below freezing, installation cost, rebates, and so on, then picking from all of those results.

And there it is. That's very likely why the 76% fell to 38%. Ahrefs' own reading of their update was that AI Overviews seem to be relying less on the results for the original query and more on the sources showing up in fan-out query results. Their January 2026 note that AI Overviews had moved to Gemini 3 fits that too. Careful here, that's their interpretation and mine, not something Google has confirmed in those words. But it's the most plausible explanation I've seen, and it has a big practical consequence.

You might be cited for a question you never targeted.

Your page about cold-weather heat pump performance can get pulled into an answer about whether heat pumps are worth it, because the system fanned out to your sub-question. Equally, your beautifully optimized page for the main query can be skipped, because the system found a better passage elsewhere for one of the sub-questions.

Ahrefs saw something similar outside Google. In an August 2025 study they took 15,000 long-tail queries, ran them through Google and Bing, and asked the same questions to ChatGPT, Gemini, Copilot and Perplexity. On average, only 12% of the URLs the AI assistants cited also ranked in Google's top 10 for the same query. About 80% of the cited URLs didn't rank anywhere in Google for that query. Perplexity was the outlier, with nearly 1 in 3 of its citations ranking in the top 10.

Read that twice. Most of what these assistants cite doesn't rank for the question you'd think it's answering.

Now, does that mean ranking doesn't matter?

I don't think so, and here's where it gets interesting, because the research really does disagree.

A 2025 paper called C-SEO Bench, from Parameter Lab, NAVER AI Lab and university collaborators, built a benchmark across six domains (retail products, video games, books, web questions, news and debate) and tested whether rewriting documents to please the language model, the kind of tricks the GEO paper proposed, actually moved a document up in the model's citations.

Mostly it didn't.

But they also tested something simpler. What if you just move the document to the first position in the list of sources the model sees? That produced far bigger gains than any rewriting method, across every model they tried. Their conclusion was blunt, traditional SEO, the stuff that improves your position in retrieval, "remain[s] essential."

So you have two findings that look like they fight each other. Ahrefs says most AI citations don't rank in Google's top 10 for the original query. C-SEO Bench says position in the retrieved set is the strongest lever there is.

I think they're both right, and they fit together once you see the pipeline. Ranking matters a lot. It's just ranking for the queries the system actually runs, which, after fan-out and rewriting, often aren't the query the user typed. You're competing in retrieval for sub-questions you can't see.

That's the honest state of things.

There's a second split in the pipeline that trips people up constantly. Training versus retrieval.

A language model learns a huge amount from web text during training. That's frozen at some cutoff. Separately, when it answers with live search, it retrieves pages right then. These are different doors into the model, and the companies run different crawlers for them.

OpenAI's documentation is a clear example. GPTBot crawls content that "may be used in training" its foundation models. OAI-SearchBot is used "to surface websites in search results in ChatGPT's search features." The settings are independent. You can block GPTBot (no training) and allow OAI-SearchBot (still appear in ChatGPT search). OpenAI says sites that opt out of OAI-SearchBot won't be shown in ChatGPT search answers, though they can still appear as navigational links.

Anthropic splits things similarly into ClaudeBot (training), Claude-User (fetches when a user asks something) and Claude-SearchBot (search quality). Perplexity has PerplexityBot and Perplexity-User. Google has Google-Extended, which controls use in Gemini and some other systems but doesn't affect Google Search, AI Overviews included, where regular Googlebot rules apply.

Why does the training door matter if search is where citations come from?

Because not every answer uses search. Ask a chat model a general question with browsing off, or in a mode where it decides it doesn't need to look anything up, and it answers from what it absorbed in training. There are no citations in that case, and nothing you publish today shows up until some future model is trained on it.

For an old, well-covered brand, that memory is probably decent. For a new one it's close to empty. I'd guess this is why a lot of small companies get a confident "I'm not familiar with that company" from a chatbot and assume they've been penalized. They haven't. They just aren't in the memory yet, and the question didn't trigger a search.

So there are really two different visibility problems hiding under one word. Being in the model's memory is slow, indirect, and mostly a function of how much the wider web has written about you over time. Being in the model's live retrieval is faster and much closer to SEO. When people argue about whether GEO "works," they're often talking about different doors without noticing.

I checked a few big publishers' robots.txt files today to see how they handle this. The New York Times disallows GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, CCBot and Applebot-Extended. The BBC disallows a similar list. Those are deliberate business decisions, often tied to licensing and lawsuits, and that's their call.

But I'd bet a fair number of smaller sites have copied a "block all AI bots" snippet from a blog post without realizing they just removed themselves from ChatGPT search too. That's the kind of thing part 3's audit catches.

Here's the part nobody likes hearing. Even when everything is set up correctly, the answers can be wrong about you.

In March 2025, the Tow Center for Digital Journalism at Columbia tested eight AI search tools on a deliberately simple task. Give the tool an excerpt from a news article, ask it to identify the article's headline, publisher, date and URL. 1,600 queries. Collectively the tools gave incorrect answers to more than 60% of them. Perplexity was wrong 37% of the time, Grok 3 was wrong 94% of the time.

Some cited syndicated copies instead of the original, for example ChatGPT citing a Yahoo News republication of a USA Today article even though USA Today blocks ChatGPT's crawler. And several tools answered confidently rather than declining.

I don't think that study proves the engines are useless. It was a narrow task, and things have shifted since. What it proves is that the pipeline has failure modes at every step. Retrieval can grab the wrong copy, generation can mash sources together, attribution can point at the wrong URL.

If you're a brand, some of what's said about you in AI answers will be wrong in ways you didn't cause and can't directly fix. Part of GEO is making the correct version so easy to find and so consistent across the web that the wrong versions lose more often.

Let me pull this together into a model you can actually carry around. It's not perfect, but I use it.

When an AI engine answers a question that needs the web, picture four gates.

The first gate is access. Can the relevant crawler or fetcher reach your page at all? robots.txt, firewalls, CDN bot rules, login walls, content that only renders with heavy JavaScript.

The second gate is retrieval. For the queries the system actually runs, including the fan-out sub-queries, does your page come back in the candidate set, and how high? This is where classic SEO lives, and C-SEO Bench says it's the strongest lever.

The third gate is selection. Out of the candidates, which passages does the model actually use? This is where clarity, specificity, verifiable facts and good structure may help. Part 4 is about this gate, and I'll warn you now, the evidence here is the shakiest.

The fourth gate is attribution. Does the final answer name you, link you, and describe you correctly? This is where the Tow Center's failures live, and where third-party consistency (part 6) matters.

Most GEO advice you'll read is about gate three. Most real problems I'd expect at gates one and two. That's not a finding, that's my opinion, and the condition on it is this. If you're a well-known brand with a mature site, gates three and four probably matter more for you. If you're small or new, you're almost certainly losing at gates one and two first.

Otlume, by the way, is the second case. It's new, its knowledge base page says "Nothing here yet," and that's fine, it's where everyone starts. What it does have is gate one handled, its robots.txt explicitly allows 43 named AI and search agents and it publishes llms.txt, JSON-LD and a sitemap. That gets you in the door. It doesn't get you retrieved for category queries, as I found in part 1 when a category prompt returned other products.

Being reachable and being chosen are different gates.

There's a reason I keep using the word gate. In 1950 the communication researcher David Manning White studied a newspaper wire editor he called "Mr. Gates," and found that the editor's personal judgments about which wire stories to run shaped what an entire town read. Gatekeeping theory grew out of that.

AI answer engines are a new kind of gatekeeper, one that runs millions of times an hour, makes slightly different calls each time, and can't tell you why. You can't interview it. You can only study its outputs, at scale, and adjust what you feed it.

That's the honest job description. Not hacking the gatekeeper. Studying it.

In part 3 we'll do exactly that, an audit of where you stand, built so that the randomness of these systems doesn't fool you. Bring the prompts you ran after part 1.