BM25
Quick Take
- In brief: BM25 ranks documents by matching query terms.
- Useful for: Exact names, error codes, versions, and code symbols.
- Main limit: It does not infer that different words have similar meanings.
Overview
Searching for E_CONN_RESET should return documents that contain that exact error code. BM25 gives more weight to useful, less common query terms than to words that appear in many documents.
Core Definition
BM25 is a term-ranking function that estimates document relevance from query-word frequency, document frequency, and document length.
Background and Evolution
BM25 comes from information retrieval and remains useful in modern AI search. It can provide exact lexical matches that semantic retrieval may miss.
How It Works
The ranking score considers how often a query term appears in a document, how common it is across the collection, and how long the document is. Search systems may adjust the parameters for their data.
Applications in Software Development and Testing
Use BM25 to find an exact API name, version, stack trace, or test identifier in technical documentation. It can also supply candidates to a retrieval-augmented generation system.
Strengths and Limitations
BM25 is fast and its matches are easy to inspect. Different wording, spelling, tokenization, or language can prevent a match.
Comparison with Similar Terms
Vector search can find passages with related wording. BM25 favors literal term matches. Hybrid search combines both result sets.
Best Practices
Test BM25 on real queries that include both ordinary questions and exact identifiers. Check how the search engine tokenizes punctuation and code symbols.
Common Pitfalls
Do not assume lexical search is obsolete because a system uses embeddings. Error codes and function names often need exact matching.
FAQ
Is BM25 still useful when vector search is available?
Yes, when exact terms matter. Compare real-query results and operating cost before choosing one retriever or combining them.