Rerankers in AI Search: How Your Best Passage Gets Picked

AI search engines pick passages, not pages, and they do it in stages. They expand your question into several searches, pull a broad set of candidate passages with keyword (lexical) and embedding (dense) retrieval, then a reranker, usually a cross-encoder model, reads the question and each passage together and scores how well that passage answers it. Only the top-scoring passages reach the language model that writes the answer and picks the citations.

This guide walks through that pipeline using what search vendors actually document (Perplexity, Microsoft, Google Cloud, OpenAI, Anthropic), marks what is documented and what is inferred for each AI engine, and then turns each stage into writing rules. We also ran our own passages through two open-source rerankers on October 24, 2026, so the before and after rewrites below come with real scores, including one result that surprised us.

How does AI search choose which passages to cite?

AI search chooses passages through a funnel: cheap methods cast a wide net, expensive methods judge a short list, and the language model works only from what survives. The table below lays out the stages with a documented example of each, so you can see which parts are public engineering and which parts are guesswork.

Stage	What happens	Documented example	What it means for your page
1. Query planning	The prompt is rewritten and often split into several searches (query fan-out).	Google says AI Mode issues "a multitude of queries simultaneously" and Deep Search "can issue hundreds of searches". OpenAI's retrieval API has a rewrite_query option. Azure's semantic ranker can create up to 10 query variants.	Your passage competes on sub-questions, not just the head query.
2. Candidate retrieval	Fast keyword (BM25) and embedding (vector) searches each return a long list.	Perplexity: "We do not force a choice between lexical and semantic retrieval" and queries "both modalities".	The passage needs the words a searcher uses and the meaning they intend.
3. Fusion	The lists are merged, often with Reciprocal Rank Fusion (RRF).	Azure AI Search scores each result as 1/(rank + k), with k typically 60. OpenAI lets developers weight embedding vs keyword matches inside RRF.	Ranking reasonably well in both lists beats ranking first in one.
4. Reranking	A slower, more accurate model scores each query-passage pair and reorders the short list.	Perplexity uses "cross-encoder reranker models to perform the final sculpting of the result set". Azure's semantic ranker reranks the top 50.	This is where a specific, self-contained answer beats a vague one.
5. Generation and citation	The language model reads the top passages, writes the answer and attaches links.	Google: a page must be "indexed and eligible to be shown in Google Search with a snippet" to be a supporting link.	Snippet controls and answer position inside the context window still matter.

Stage 1 is the subject of our separate explainer on query fan-out. This article covers stages 2 to 5: how candidates are found, how they are scored, and why one paragraph wins over another.

Why not let the language model read everything?

Cost and speed. Google Cloud's Vertex AI RAG Engine documentation lists its dedicated ranking API at under 100 milliseconds per request, and its alternative LLM reranker, which makes a separate Gemini call to judge relevance, at 1 to 2 seconds. A search engine answering millions of questions cannot run a large model over thousands of pages per query, so it uses cheap retrieval to find candidates and a compact reranker to judge them.

What is the difference between lexical and embedding retrieval?

Lexical retrieval matches words; embedding retrieval matches meaning. Most production AI search systems run both and merge the results, because each one misses things the other catches.

• Lexical retrieval (BM25) scores a passage by how many of the query's words it contains, gives rare words more weight than common ones, and adjusts for length. It is fast and precise for names, product codes and exact terms, and blind to synonyms. A passage that says "re-scoring model" will not match a search for "reranker".
• Embedding (dense) retrieval converts the query and every passage into vectors, lists of numbers positioned so that similar meanings sit close together, then returns the nearest neighbors. It catches paraphrases, but it can blur exact identifiers and rare terms. The Dense Passage Retrieval paper (Karpukhin et al., EMNLP 2020) showed a dense retriever beating a strong BM25 system by 9 to 19 points of top-20 passage accuracy on open-domain question answering, which is why dense retrieval became standard.

	Lexical (BM25)	Embedding (dense)
Matches on	Shared words	Similar meaning
Strong at	Brand names, model numbers, exact terms, rare words	Paraphrases, natural-language questions, synonyms
Weak at	Synonyms and reworded questions	Exact identifiers and very specific terms
Writing implication	Use the words your reader would type, including the entity name	Make the passage clearly about one idea so its vector is not diluted

Fusion then combines the two lists. Microsoft's documentation for Azure AI Search spells out the standard method: each result gets 1/(rank + k) from every list it appears in, with k usually set to 60, and the scores are summed. A passage ranked 3rd in both lists outscores one ranked 1st in one list and absent from the other. OpenAI's retrieval guide exposes the same idea as tunable embedding_weight and text_weight settings for "reciprocal rank fusion".

What is a reranker, and what does it actually score?

A reranker is a second-stage model that takes the query and one candidate passage as a single input and outputs a relevance score. It runs only on the short list that retrieval produced, and it reorders that list so the best answers move to the top.

The difference from retrieval is architectural. Retrieval models are bi-encoders: they turn the query and the passage into vectors separately, which lets an engine pre-compute vectors for billions of passages. A cross-encoder reads the query and the passage together, so every word in the passage can be compared with every word in the question. That is far more accurate and far too slow to run on a whole index. The approach goes back to "Passage Re-ranking with BERT" (Nogueira and Cho, 2019), which reranked BM25 candidates with BERT and beat the previous best result on the MS MARCO passage leaderboard by 27% (relative, MRR@10). Commercial rerankers such as Cohere Rerank and Google's ranking API follow the same pattern: query plus documents in, relevance scores out.

What does a high reranker score look like?

Microsoft publishes the clearest public definition. Azure AI Search's semantic ranker, which uses "deep learning models adapted from Microsoft Bing", gives every document a score from 0 to 4 and documents what each level means:

Score	Microsoft's definition
4.0	Highly relevant and answers the question completely, though the passage might contain extra text unrelated to the question.
3.0	Relevant but lacks details that would make it complete.
2.0	Somewhat relevant; answers the question partially or only addresses some aspects.
1.0	Related to the question, and answers a small part of it.
0.0	Irrelevant.

Notice what is missing: authority, links, brand and word count. The scale rewards one thing, whether the passage answers the question, and it separates "related to the topic" (1.0) from "answers it completely" (4.0). That gap is where most content loses. A paragraph can be accurate, on-topic and well written and still score like a 1 because it talks around the question.

Why do rerankers favor self-contained, specific passages?

Because the reranker sees only the text it is given, and the amount it is given is capped. Three documented limits show this:

• Azure's semantic ranker assembles up to 2,000 tokens per document from title, keywords and content fields (128 tokens for the title), and "ignores anything after the maximum limit". Only the top 50 results are reranked at all.
• Google Cloud's ranking API accepts 1,024 tokens per record on its current models (512 on version 003 and earlier). If the title and content exceed that, "the extra content is truncated".
• Many open-source rerankers, including the two we tested below, read at most 512 tokens for the query and passage combined.

Put together: the reranker does not know what your previous paragraph said, cannot follow "as mentioned above", and may never read your conclusion if it sits past the token limit. A passage that names its subject, states the answer early and carries its own context is the only kind that scores well under those conditions.

Which AI search engines use rerankers?

Of the major consumer AI search engines, Perplexity is the one we found that has publicly described cross-encoder reranking in its web search stack. For the others, the honest answer is that the overall pattern is documented in their developer products, and the consumer pipelines are not. This table separates the two.

Engine	What is documented	What is inferred, not confirmed
Perplexity	Its September 2025 engineering post describes an index of more than 200 billion URLs, hybrid lexical and semantic retrieval, early "lexical and embedding-based scorers", and "more powerful cross-encoder reranker models" at the end. Documents are split into "self-contained spans, each of which can be individually retrieved and ranked".	Which reranker models it uses, how deep the candidate list is, and how other signals are weighted.
Google AI Overviews and AI Mode	Answers are grounded in pages from Google's index, using query fan-out (AI features and your website). For classic Search, Google's October 2020 passage ranking announcement said scoring individual passages, in addition to the whole page, would improve 7% of queries across all languages. Google Cloud sells a reranker (the ranking API) publicly.	Whether AI Overviews use a separate passage reranker, and how it relates to the ranking API. Google has not published this.
ChatGPT search	OpenAI's developer retrieval tools document query rewriting, keyword plus semantic search, RRF fusion and default 800-token chunks with 400-token overlap.	That consumer ChatGPT search works the same way. See our breakdown of how ChatGPT search works for what OpenAI has confirmed.
Microsoft Copilot	Azure AI Search's semantic ranker uses models "adapted from Microsoft Bing".	How Copilot's own answers select passages from Bing results.
Claude	Anthropic's Contextual Retrieval research (September 2024) showed reranking cutting retrieval failures in its own tests.	How passages are ranked inside Claude's web search tool. Anthropic has not published this.

The practical point: you do not need to know each engine's exact model. Every documented system follows the same retrieve-then-rerank shape, and the writing rules that help in one help in all of them.

How do AI search engines split pages into chunks, and why do headings matter?

Retrieval systems split each page into chunks, often a few hundred tokens long, and score each chunk separately. The engine decides the cut points, not you. OpenAI's file search, for example, defaults to 800-token chunks with a 400-token overlap, and developers can set anywhere from 100 to 4,096 tokens. Perplexity describes splitting documents into self-contained spans. No consumer AI search engine publishes its web chunk size, so any article quoting an exact "optimal chunk length" for ChatGPT or Google is guessing.

Headings matter because they are often the only context a chunk carries about its subject. Anthropic's Contextual Retrieval study used the example chunk "The company's revenue grew by 3% over the previous quarter", which does not say which company or which quarter, so it fails to match a question about ACME Corp's Q2 revenue. Adding a short context note to each chunk cut the top-20 retrieval failure rate from 5.7% to 3.7% (35% fewer failures) with embeddings alone, to 2.9% when contextual BM25 was added, and to 1.9% with a reranker on top (67% fewer). In that setup, retrieval pulled the top 150 chunks and the reranker kept the best 20.

You cannot add context notes to Google's index, but you can write sections that already contain their context: a descriptive heading, the entity named in the first sentence, and no pronouns that point to a previous section. Our test below shows how much that one change moves a reranker score.

Should you chunk your content for AI search?

No, not in the sense of breaking pages into fragments. Google's guide to optimizing for generative AI features says "There's no requirement to break your content into tiny pieces for AI to better understand it", and on the Search Off the Record podcast in January 2026 Danny Sullivan was blunter about "bite-sized chunks" for LLMs: "So we don't want you to do that" (reported by PPC Land).

That advice and the reranker mechanics do not conflict. Google is warning against fragmenting pages into thin, machine-targeted pieces. The reranker evidence argues for something readers also want: each section answers its heading clearly, early and without depending on the section before it. Write full pages for people, and make every section stand on its own.

What happened when we ran real passages through two rerankers?

We built a small retrieval pipeline on October 24, 2026 to see how specific writing choices move scores at each stage. It is a demonstration of mechanisms, not a prediction of any engine's behavior, but every number below is a measured output, not an estimate.

Method. We wrote 19 passages: pairs or triples of "before" and "after" versions for five test questions, plus five on-topic distractors (about embeddings, BM25, query fan-out, schema history and RRF). All 19 competed in one pool for every question. For each question we recorded the BM25 rank (rank_bm25), the embedding rank (all-MiniLM-L6-v2, cosine similarity), the RRF-fused rank (k = 60), and scores from two open-source cross-encoder rerankers: ms-marco-MiniLM-L6-v2 (a small model trained on MS MARCO, raw score where higher is better) and bge-reranker-base (a larger model, score from 0 to 1). Both read at most 512 tokens per query-passage pair. Scores are not comparable between the two models, only within each one.

Test (question)	Passage version	BM25 rank	Embedding rank	MiniLM reranker score (rank)	BGE reranker score (rank)
1. "what does a reranker do in AI search"	Before: scene-setting intro	9	4	-4.41 (7)	0.003 (12)
	After: direct definition	1	1	9.74 (1)	0.9998 (1)
2. "what chunk size does OpenAI file search use by default"	Before: numbers promised "further down"	2	2	0.97 (2)	0.026 (2)
	After: numbers in first sentence	1	1	9.26 (1)	0.9998 (1)
3. "how many results does the Azure AI Search semantic ranker rerank"	Before: pronoun-only ("It only looks at the top 50")	16	11	-11.30 (14)	0.014 (5)
	Same text with heading restated	2	2	4.75 (2)	0.958 (2)
	After: fully self-contained	1	1	9.23 (1)	0.9998 (1)
4. "does Google require structured data for AI Overviews" (695-word passage, about 844 tokens)	Answer at the end	3	6	-3.57 (4)	0.414 (4)
	Same text, answer first	4	1	7.91 (2)	0.994 (3)
5. "how do AI search engines choose which passages to cite"	Keyword-stuffed, no information	1	1	9.56 (2)	0.999 (2)
	Correct answer, paraphrased away from the question's words	14	4	-8.39 (11)	0.023 (7)
	Correct answer in the question's own words	2	2	10.55 (1)	0.9999 (1)

Ranks are out of 19 passages. In test 4, a shorter 368-word version of the same passage (about 449 tokens, inside the 512-token limit) let the larger BGE model score both orders almost equally (0.999 vs 0.998), while the small MiniLM model still preferred answer-first (8.45 vs 4.04). Once the passage ran past the limit, the answer at the end was cut off before either model read it.

What the results show

1. Direct answers win by a wide margin. In tests 1 and 2, the only change was moving from talking about the topic to answering the question. The BGE score went from near zero to near one in both.
2. Context-free passages can fall out at retrieval, before reranking. The pronoun-only passage in test 3 ranked 16th of 19 on BM25 and 15th after fusion. In a real engine with millions of candidates, it would likely never reach a reranker. Restating the heading inside the passage brought it to 2nd on every measure.
3. Truncation is real. When the answer sat past the 512-token window, both rerankers scored the passage far lower, because they never saw the answer.
4. Small rerankers reward lexical overlap, even when it is empty. This was the surprise. A stuffed passage that repeated the question and said nothing scored 0.999 on BGE, while a correct answer written in different words scored 0.023. This matches peer-reviewed findings: Hagström et al. (FEVER 2025) found language model rerankers "struggle to outperform a simple BM25 baseline" on one dataset and traced the errors to lexical dissimilarity.

Test 5 is not a case for keyword stuffing. The stuffed passage still lost to the real answer that used the question's wording, it gives a language model nothing to quote, and Google treats text written only to manipulate ranking as spam. The lesson is narrower and more useful: do not paraphrase away from the words your readers use. Answer in their vocabulary, then add the precise terms.

Before and after: three passage rewrites that score higher

Each rewrite below is taken verbatim from our test, with its measured scores. The same three edits cover most of the passages we would fix on a typical page.

Rewrite 1: replace the scene-setting intro with the answer

Before (BGE 0.003, rank 12 of 19):

Search has changed a lot over the past few years. There are many moving parts behind the answers you see in AI tools, and some of them are more important than people realize. In this section we look at one of those parts and why it matters for anyone who publishes content and wants to be visible.

After (BGE 0.9998, rank 1 of 19):

A reranker is the second-stage model in an AI search pipeline. After a fast retriever pulls a broad set of candidate passages, the reranker reads the query and each passage together and gives every pair a relevance score. Only the top-scoring passages are passed to the language model that writes the answer.

Why it works: the first sentence is a definition that could be quoted alone, and the next two sentences add the mechanism. The before version is about 60 tokens of throat-clearing, the kind of text Microsoft's rubric describes as merely related to the question: on topic, but it answers nothing.

Rewrite 2: give a pronoun-only section its subject back

Before (MiniLM -11.30, rank 14 of 19; BM25 rank 16):

It only looks at the top 50. Anything below that is never rescored, however good it is, and each one is cut down to a summary before scoring.

After (MiniLM 9.23, rank 1 of 19; BM25 rank 1):

Azure AI Search's semantic ranker reranks only the top 50 results from the initial BM25 or RRF-ranked list. Results below position 50 are never rescored, and each document is condensed into a summary of up to 2,048 tokens before the ranker scores it.

Why it works: on the page, the before version sat under a heading that explained what "it" was. Once the passage is chunked, that heading may not travel with it. Naming the product, the action and the numbers makes the passage findable by keyword, by meaning and by the reranker.

Rewrite 3: answer in the reader's words, not a synonym tour

Before (BGE 0.023, rank 7 of 19; BM25 rank 14):

Answer engines first pull a broad set of candidate text spans with fast keyword and vector lookups. A slower scoring model then reads the question alongside each span and re-orders them. The few spans that score highest are handed to the language model, which writes the reply and links the pages they came from.

After (BGE 0.9999, rank 1 of 19; BM25 rank 2):

AI search engines choose which passages to cite in two steps. Fast keyword and vector retrieval first pulls a broad set of candidate passages. A reranker then reads the question alongside each passage and re-orders them, and the highest-scoring passages go to the language model, which writes the answer and cites the pages they came from.

Why it works: both versions are accurate. The before version swaps every key term for a synonym ("answer engines", "text spans", "reply", "links"), which a writer might do to avoid repetition. Google's guide is right that its systems understand synonyms, but our small rerankers did not reward them. The after version opens by restating the question as a statement, which costs nothing and helps at every stage.

What does not matter as much as people think?

Several popular tactics either have no evidence behind them or target a stage they cannot affect.

• Exact chunk or paragraph lengths. No consumer AI search engine publishes its chunk size, and the engine picks the cut points. Figures like "write 200-word blocks" are guesses. What does matter is that the answer sits early in a section, so it survives wherever the cut falls.
• Splitting one page into many thin pages. Google's AI guide says its systems can "understand the nuance of multiple topics on a page", and mass-producing near-duplicate pages for each variation can fall under the scaled content abuse section of its spam policies.
• Schema markup and llms.txt for AI citations. Rerankers score text. Google says structured data "isn't required for generative AI search" and that llms.txt files "will neither harm nor help" visibility in Google Search. Schema remains useful for rich results.
• Keyword stuffing. It can fool a small reranker, as our test showed, but it gives the language model nothing to cite and runs into spam policies. Use the reader's words once, clearly, in a real answer.
• Reranker "scores" from third-party tools as a ranking forecast. Open models like the ones we used show direction, not outcomes. Production engines use different models plus quality, freshness and diversity signals that a single score cannot capture.
• Rewriting passages on pages that are not indexed or not retrieved. A reranker only scores what retrieval hands it. If the page is not indexed, blocked, or invisible for the sub-query, passage edits do nothing.

Why does my page rank in Google but not get cited in AI answers?

Usually because ranking and citation are decided at different stages. Your page can rank on the head query while losing a fan-out sub-query, or be retrieved but lose the reranking round to a passage that answers more directly. Peec AI's reranker guide makes the same point: diagnose the failed stage before editing. This table maps what you can observe to the stage that probably failed.

What you see	Stage that probably failed	What to fix
Page is not indexed, or has nosnippet / max-snippet limits	Eligibility (before stage 2)	Indexing and snippet controls. Google requires a page to be indexed and snippet-eligible to be a supporting link.
Competitors are cited for sub-questions your page does not cover	Query planning and retrieval	Add a section that answers that sub-question. See query fan-out.
Your page covers the sub-question, but a page with a blunter answer is cited	Reranking	Put a direct answer in the first sentence of the section, in the searcher's wording.
Your page is cited, but for a minor point, not your main answer	Reranking or truncation	Move the main answer above the background. Make the key section self-contained.
Citations come and go between runs of the same prompt	Normal variation across stages	Measure over repeated runs before deciding anything (see the data below).
Strong, direct passage but the answer cites review sites or forums instead	Generation, source diversity or trust	Earn coverage on the sources the engine prefers. Our study of what sources LLMs cite shows the patterns.

Passage-quality checklist

Use this before publishing or when refreshing a page. Each row ties a check to the mechanism behind it.

Check	Mechanism it serves	How to test it
The first sentence under each heading answers that heading	Reranking (complete answer), truncation	Read only the first sentence. Would it work as a quote?
The section names its subject (product, brand, concept) instead of "it" or "this"	Chunking, lexical and embedding retrieval	Paste the section alone into a blank doc. Is it clear what it is about?
The heading describes the content, ideally as the question a reader asks	Chunk context, fan-out matching	Could the heading be typed into a search box?
Key terms use the reader's vocabulary, with technical terms added, not substituted	Lexical retrieval, reranker overlap	Compare against the questions customers and People Also Ask use.
Specifics are in the passage: numbers, names, dates, steps	Reranking ("lacks details" scores lower)	Would Microsoft's rubric give it a 4 (complete) or a 3 (lacks details)?
One idea per section; long sections keep their answer near the top	Embedding focus, token limits	Is the answer within the first 100 to 150 words of the section?
Important text is in HTML, not in images, tabs that load later, or scripts	Retrieval (crawlable text)	View the page source or use URL Inspection in Search Console.
No snippet restrictions on pages you want cited	Eligibility	Check for nosnippet, data-nosnippet and max-snippet.
The page is current, with a visible date where it matters	Retrieval and source selection	Confirm facts and prices on the day you update.

How do you measure which of your passages get cited?

Track which of your URLs are cited for a fixed set of prompts, run those prompts repeatedly, and read each answer against your page to see which passage it used. One run tells you very little, because citations shift between runs of the same prompt.

Our own tracking data shows how much. Across 972 successful runs of 42 prompt-engine pairs tracked in AI Rank Lab (February 13 to October 22, 2026, pairs with at least five runs, engines including ChatGPT, Claude, Gemini and Perplexity), the median overlap of cited domains between consecutive runs of the same prompt was 0.41 on a 0 to 1 scale (Jaccard similarity). Across all runs of a pair, the median number of distinct cited domains was 2.7 times the number in a single run, and a median of only 3.4% of cited domains appeared in every run. That data is at domain level, so it cannot tell us how much of the variation comes from reranking rather than fan-out or generation. It does tell you not to judge a rewrite from one screenshot.

A practical workflow:

1. Pick 10 to 20 prompts where you expect your page to be a source, including the sub-questions your sections answer.
2. Run each prompt several times across engines and record which URLs are cited.
3. For each citation of your page, find the sentence the answer drew on. If it is a minor point, your main answer lost the reranking round.
4. Rewrite the losing sections using the checklist above. Change only those sections, so you can attribute the result.
5. Re-run the same prompts after the page is recrawled and compare citation rates over several runs, not one. Our guide to running GEO experiments covers baselines and sample sizes.

Tools that help with this, assessed on the same criteria (which engines they cover and whether they show cited URLs):

• AI Rank Lab. Disclosure: AI Rank Lab is our product. We list it first because this is our site, and we've kept the same criteria for every tool below. Its citation tracker shows which domains and URLs ChatGPT, Claude, Gemini, Perplexity, Google AI Overviews and AI Mode cite for the prompts you run, and stores every run so you can compare before and after a rewrite. It shows cited URLs, not the exact passage, so step 3 is still a manual read. Prompts you opt in to the Action Plan citation tracker are re-checked weekly; other prompts run when you run them.
• Google Search Console. The Generative AI performance report shows how often links to your pages appeared in AI Overviews and AI Mode, measured by Google itself. It reports impressions only, with no queries, and does not show which passage was used or how competitors fare.
• Peec AI. Tracks prompts and citations across AI engines, and its team publishes some of the most detailed research on rerankers for marketers.
• Open-source rerankers. The sentence-transformers library and models like the two we used let a technical team score draft passages against target questions in minutes, for free. Treat the scores as a direction check, not a forecast.

Methodology and limitations

Vendor documentation was checked on October 24, 2026; documentation pages change, and we quote the version live that day. The reranker test used 19 passages we wrote ourselves, five questions, and two small open-source rerankers (ms-marco-MiniLM-L6-v2 and bge-reranker-base) with all-MiniLM-L6-v2 embeddings and rank_bm25, run locally with the sentence-transformers library. Production AI search engines use larger, proprietary models, much bigger candidate pools and signals beyond relevance, so the test shows how writing choices move relevance scores, not whether any engine will cite a page. The first-party citation data is aggregated and anonymized across AI Rank Lab accounts and measures domains, not passages. Where this article describes an engine's internals, it says whether the source is the engine's own documentation or an inference.

The short version

AI search finds candidate passages with keyword and embedding retrieval, then a reranker decides which ones the language model gets to read. Rerankers see only a capped slice of each passage, reward complete answers over related discussion, and, at least in the models we tested, lean heavily on shared wording. So write each section to answer its heading in the first sentence, in your reader's words, with its subject named and its specifics included. If you want to see which of your URLs AI engines actually cite before and after you make those edits, you can track citations in AI Rank Lab.