BM25, explained

01

What is BM25?

BM25 is a ranking function that scores how well a passage of text matches a search query. It adds up evidence from each word of the query: how often the word appears in the passage, how rare it is across the collection, and how long the passage is. The highest score ranks first.

The name stands for Best Matching 25. It comes from the Okapi retrieval system at City University London, where Stephen Robertson, Steve Walker and colleagues tested a series of weighting functions; BM25 first appeared in their experiments for TREC-3, the third Text REtrieval Conference, in 1994 2. Its theory is the probabilistic relevance framework that Robertson, Karen Spärck Jones and others developed from the 1970s onwards, which Robertson and Hugo Zaragoza set out in full in 2009 1.

More than three decades later, BM25 is still the reference point for lexical search, the family of methods that match the words of a query against the words of a document. It needs no training data and no neural network, and every part of a score can be traced to a count. New retrieval models are routinely measured against it, and on the BEIR benchmark it proved a more robust baseline across unfamiliar collections than many trained retrievers 7.

Brello 1.0 uses BM25 for one job: choosing which passages of the web pages it has just read are passed to the language model on the phone (section 06).

02

The BM25 formula, term by term

BM25 gives a passage one contribution for each query word and adds them up. Each contribution is the word’s rarity, its inverse document frequency (IDF), multiplied by a term-frequency factor that rises with each repetition, levels off, and is scaled down for longer passages. Equations 1 to 3 state it in full.

Inverse document frequency: how rare the word is

IDF rewards words that pick out few passages. A word found in every passage tells the ranker almost nothing, so its weight falls close to zero; a word found in only a few separates them sharply. The idea of weighting a search term by its rarity goes back to Karen Spärck Jones in 1972 3.

The 1 inside the logarithm of Equation 2 is a practical refinement. Without it, the weight is ln((N − n + 0.5) ÷ (n + 0.5)), the form given by Robertson and Zaragoza 1, which turns negative for any word found in more than half the passages, so matching a common word would lower a score. Elasticsearch adds the 1 and keeps every weight positive 5. The examples on this page use that form.

The term-frequency factor: how often the word appears

The factor in Equation 3 grows with f, the number of times the word occurs in the passage, but with diminishing returns. Because f appears in both the numerator and the denominator, the factor climbs steeply for the first occurrence and then flattens towards a ceiling of k₁ + 1. Table 1 gives its values at k₁ = 1.2 and b = 0.75, the settings Brello 1.0 uses: in a passage of average length, the first occurrence is worth 1.000, the second adds 0.375 and the third adds 0.196, and ten occurrences reach 1.964 of a possible 2.2.

Length normalisation: how long the passage is

The ratio |D| ÷ avgdl compares the passage’s length with the average. A long passage has more chances to contain any word by accident, so BM25 enlarges the denominator for passages longer than average and shrinks it for shorter ones; b sets how strongly. At b = 0.75, a single occurrence counts 1.257 in a passage half the average length and 0.710 in one twice the average length.

03

What k1 and b control

k₁ sets how quickly repeated occurrences of a word stop adding to a passage’s score, and b sets how much a passage’s length counts against it. Both are chosen by whoever runs the search; neither is learned from the query.

At k₁ = 0 the factor is exactly 1 for any word that is present, so BM25 only asks which of the query’s words a passage contains. As k₁ grows, each repetition counts for more, and for very large values the factor grows almost in proportion to the count, as raw term frequency does in classic TF-IDF. b runs from 0, where length is ignored, to 1, where a passage’s matches are scaled fully by its length relative to the average. The middle ground reflects two reasons a passage can be long: it may say the same thing in more words, which deserves to be normalised away, or it may cover more ground, which does not.

Typical settings sit in a narrow band. Manning, Raghavan and Schütze report that experiments support k₁ between 1.2 and 2 and b = 0.75 4, and Elasticsearch uses k₁ = 1.2 and b = 0.75 unless it is configured otherwise 5. Table 2 summarises the extremes.

Figure 1 puts both parameters in your hands. It scores six short passages against the query ‘why is the sky blue’, the search text Brello 1.0 writes for the question “Why is the sky blue? Keep it short.”, and draws each word’s contribution as a coloured segment. Play the steps, or drag the sliders and watch the ranking re-sort.

04

BM25, TF-IDF and embeddings

TF-IDF and BM25 both score a passage by the words it shares with the query; BM25 adds saturation and an adjustable length correction. Embedding-based retrieval works differently: a neural network turns the query and each passage into vectors, and passages are ranked by how close their vectors are, which lets them match on meaning rather than spelling.

Classic TF-IDF multiplies a word’s count by its IDF, often damping the count with a logarithm and normalising long documents by the length of their vectors 4. BM25 grew out of a probabilistic model of relevance rather than vector geometry, and its two parameters make saturation and length explicit 1. Both are sparse methods: a passage is represented by the words it contains, so an inverted index can find every passage that contains a query word without reading the rest.

Dense retrieval, such as Dense Passage Retrieval 6, trains encoders so that questions and the passages that answer them land close together in vector space. It can find a passage that shares no words with the question. The costs are a trained model, an index of vectors and the computation to embed every passage, and quality can drop on collections unlike the training data: in BEIR, BM25 was a robust zero-shot baseline that dense retrievers often failed to beat outside their training domain 7. Hybrid systems combine the two, using sparse scores for exact names, numbers and rare terms and dense scores for paraphrase. Table 3 compares the three.

05

A worked example

Scoring one passage by hand shows how the parts combine. Take the short explainer from Figure 1, “Why is the sky blue? Sunlight is scattered by the molecules in air, and blue light is scattered far more strongly than red.”, against the query ‘why is the sky blue’ with k₁ = 1.2 and b = 0.75. Its five word contributions add up to 2.61.

Start with the collection. There are N = 6 passages with an average length of 34.67 words (208 words in all), and the explainer has 23. Its length term is 1 − 0.75 + 0.75 × 23 ÷ 34.67 = 0.7476, so the denominator of Equation 3 adds k₁ × 0.7476 = 0.8971 to each word’s count. Table 4 then applies Equations 2 and 3 to each query word.

Three things stand out. ‘sky’ and ‘blue’, found in only three of the six passages, supply 1.86 of its 2.61 points. ‘is’ and ‘the’, found in every passage, add about 0.24 between them, so BM25 discounts them without a list of stop words: rarity does the work. And the second ‘blue’ is worth less than the first, because the factor for two occurrences is 1.519, not twice 1.160.

Figure 1 runs the same arithmetic for every passage each time a slider moves, and its “Show data” table lists the results.

06

BM25 inside an on-device answer engine

Brello 1.0 uses BM25 to decide which parts of the web pages it has just read are given to the language model on the phone. BM25 suits the job because it needs no trained model, no server and, at the scale of a few pages, no index built in advance: the passages are scored on the phone, moments after they are fetched.

The coverage bonus counters a weakness visible in Figure 1: plain BM25 can rank a passage highly for repeating one rare word, as the paint shop page does with ‘blue’, and rewarding passages that match more of the question favours the ones that address all of it. The lead-paragraph bonus favours the opening of a page, which often states its subject. The two-per-source limit stops one long page from filling the budget, so an answer can draw on several sources.

The budget caps how much of the model’s 4,096-token context window the passages can take; the window also has to hold the instructions, up to six earlier messages and the answer (Context windows, explained). BM25 is the ranking stage of a retrieval-augmented generation pipeline. Retrieval-augmented generation, explained covers the other stages, and the research note ‘Answering from the open web, without a server’ describes the whole system.

07

The limits of lexical ranking

BM25 compares spellings, not meanings, so it misses relevant passages written in other words and can favour irrelevant ones that repeat the right words. It also cannot tell whether a passage is true.

• Vocabulary mismatch. A passage that says ‘azure heavens’ scores nothing for ‘sky’ or ‘blue’, however well it answers the question. Query rewriting and dense retrieval are the usual remedies (section 04).
• Word forms. Without stemming, which reduces words to a shared root, ‘skies’ and ‘bluest’ do not match ‘sky’ and ‘blue’. Stemming helps but can merge unrelated words, such as ‘universe’ and ‘university’.
• Word order and negation. BM25 treats a passage as a bag of words: ‘sky blue’ and ‘blue sky’ score the same, and ‘not blue’ still matches ‘blue’.
• Repetition. Saturation limits what repetition can earn, but a page stuffed with a query’s words can still outrank a better one, as the paint shop page does at k₁ = 3.
• Small collections. IDF is only as reliable as the passages it is measured on. Over a few pages, a word can look rare by chance: ‘why’ weighs 0.442 in Figure 1 only because two of the six passages lack it, and it would weigh far less across a large collection.
• Relevance is not accuracy. A passage can match every word of a question and still be wrong, out of date or written to mislead, including text crafted to give instructions to the model that reads it. Ranking decides what the model sees, not whether it is true (Why AI makes things up; Prompt injection, explained).

Brello 1.0 adds checks around BM25 rather than replacing it. Before ranking, it moves on to the next search provider unless at least a third of the top results mention the question’s key terms, which is how it rejects junk such as dictionary pages for ‘why’ when the question was ‘why is the sky blue’. After ranking, the model is told: “If the results do not answer the question, say so briefly and answer from general knowledge.” Neither check can confirm that a source is correct, and the on-device model, far smaller than frontier cloud models, can still be wrong.

Lexical ranking’s strengths mirror these limits. It matches exact names, numbers and rare terms that embedding models can blur, its scores can be explained word by word, as in section 05, and it needs no model of its own, which matters on a phone that is already running a language model.