Hybrid retrieval fusion: RRF, convex combination, and rerankers

Hybrid retrieval means running a lexical retriever (BM25) and a semantic retriever (dense vectors, sometimes learned sparse vectors) on the same query and combining their results. The case for doing it is strong and well measured. On BEIR, the zero-shot benchmark most papers use, BM25 is a "robust baseline" that many dense models fail to beat out of domain (Thakur et al., 2021). Chen et al. (ECIR 2022) found that a simple lexical-plus-dense hybrid beat the dense model by about 20% and BM25 by about 9.5% on average on three out-of-domain datasets: the two retrievers are complementary even when one of them is weaker.

The hard part is the combination step, and it's the part people tune least. This post covers the three families of fusion functions, the normalization pitfalls that come from BM25 and cosine living on different scales, what the RRF constant k does, how to evaluate a fusion choice without fooling yourself, and the production details that decide whether hybrid search behaves well under filters and pagination. The last third is about Loams: what the engine implements, taken from the code on the dev branch, and what it doesn't do yet.

Three families of fusionCopy Anchor Link

Rank-based fusion looks only at positions. Reciprocal Rank Fusion (RRF), from Cormack, Clarke and Büttcher (SIGIR 2009), gives each document the sum, over every input ranking, of 1 / (k + r(d)), where r(d) is its 1-based rank in that ranking. It needs no training data and no knowledge of the score scales. The paper's claim is in its title: RRF beat Condorcet Fuse, CombMNZ and the best individual system on TREC runs, and as a meta-learner it ranked LETOR 3 better than any previously reported method.

Score-based fusion normalizes each retriever's scores and then adds them, usually as a convex combination, α · s_sem + (1 − α) · s_lex. Weaviate's alpha, Pinecone's client-side hybrid_score_norm, OpenSearch's normalization processor and Elasticsearch's linear retriever all belong here. Distribution-based fusion, such as Qdrant's DBSF, is the same idea with a statistical normalization.

Learned fusion and reranking puts a model in the loop. That can be a learning-to-rank model over features from both retrievers (Vespa's phased ranking is built for this), or a cross-encoder that rereads the query together with each candidate's text and scores the pair. Cross-encoders are the strongest of the three: Nogueira and Cho (2019) improved the MS MARCO passage state of the art by 27% relative MRR@10 by reranking with BERT, and BEIR reports that reranking and late-interaction models give the best zero-shot results on average, "at high computational costs". A reranker doesn't replace fusion, though. It reorders the candidates that fusion hands it, so the recall of the fused list caps it.

What the comparison literature foundCopy Anchor Link

The most cited comparison is Bruch, Gai and Ingber, "An Analysis of Fusion Functions for Hybrid Retrieval" (ACM TOIS, doi:10.1145/3596512). The setup is a useful template. BM25 (via PISA) and all-MiniLM-L6-v2 with exact inner-product search each retrieve their top 1,000. The evaluation covers MS MARCO passage (6,980 dev queries) and eight BEIR datasets. NQ, Quora and MS MARCO count as in-domain, because the embedding model was trained on them; the other five are zero-shot. The findings:

• Convex combination beats RRF, in domain and out of domain. Their best variant, TM2C2 (a convex combination over theoretical min-max normalized scores, α = 0.8), beat RRF at k = 60 on every dataset, for example 0.454 against 0.425 NDCG@1000 on MS MARCO and 0.699 against 0.675 on HotpotQA.
• The normalization choice matters less than people think for a convex combination. Min-max and z-score mostly move the optimal α around. TM2C2's optimum was the most stable across datasets.
• CC is sample efficient. With less than 5% of the training data, often a small set of queries, TM2C2's α converged regardless of domain shift.
• RRF is sensitive to its parameters, and tuning it doesn't transfer. With a separate constant per retriever, NDCG "swings wildly". Tuned constants that helped on MS MARCO (0.451 against 0.425) hurt on HotpotQA (0.621 against 0.675).
• RRF throws information away. Two documents with cosine 0.84 and 0.40 at ranks 1 and 2 count the same as two at 0.84 and 0.839. The authors find that a smoothed RRF, which keeps some score information, did better than plain RRF, particularly out of domain.

Two caveats before you take "CC beats RRF" as settled. First, the headline numbers are NDCG@1000 (NDCG@100 for SciFact and NFCorpus), a deep-ranking metric, not the NDCG@10 most leaderboards report. Second, the gains are a few points of NDCG, and they assume you have some labelled queries to set α. RRF's real advantage is that it is hard to get badly wrong with no labels at all. Chen et al. reached the opposite conclusion from Bruch et al. on their datasets, and Bruch et al. say so. The honest summary: with even a small labelled set, a tuned convex combination over sensibly normalized scores usually wins; with none, RRF at k = 60 is a safe default.

Normalization pitfallsCopy Anchor Link

Score-based fusion assumes the two scores mean comparable things after normalization. They don't start out that way.

BM25 is unbounded and query-dependent. Its scale grows with the number of query terms and their IDF. A three-term query over a large corpus can top out above 20, and a one-term query with a common word may top out at 3. There is no fixed maximum to divide by.

Cosine is bounded, but it's used in a narrow band. Many embedding models put most query–document similarities between about 0.6 and 0.9. Within one query's top 10, the spread is often a few hundredths. So if you add raw BM25 and raw cosine, BM25 decides everything: in the appendix's example the raw sum ranks exactly in BM25 order and drops every vector-only hit from the top five. Weighting doesn't save you, because the right weight depends on the query's BM25 scale, which changes from query to query.

Min-max over the returned list stretches noise into signal. Mapping each list to [0, 1] fixes the units, but it has three failure modes that matter in practice:

1. The last hit of every list gets 0, whatever its absolute quality. A cosine of 0.811, a perfectly good match, becomes 0.18 in the example, and that document falls from second to fifth after fusion.
2. The result depends on how many candidates you fetched. Ask the vector retriever for 20 instead of 5 and the minimum drops, so every normalized score in the list changes. Top-k is no longer just a performance knob.
3. One outlier at the top compresses everything below it. A BM25 hit on an exact identifier at 40 squashes the rest of the list toward 0.

Theoretical min-max (TM2C2) replaces the observed minimum with the function's infimum: 0 for BM25 and −1 for cosine. That fixes failure 1 and part of failure 2, but because cosine's useful band is so narrow, it then gives every vector hit nearly the same normalized score. That is one reason Bruch et al.'s best α was 0.8 toward the semantic side. z-score and Qdrant's DBSF (mean ± 3σ mapped to [0, 1], not clipped) use the shape of the distribution rather than its extremes, which helps with outliers, but they are estimated from the same few returned hits, so they are noisy for small k. Qdrant's docs recommend a larger prefetch limit when DBSF rankings look unstable.

Documents missing from one list need a value. A document BM25 found but the vector retriever didn't return is not "cosine 0". It is "cosine below this list's cutoff". Every implementation silently assigns it the floor (0 after min-max). That is the right call for RRF, where absence means no vote, and a debatable one for score fusion. The larger the per-signal top-k, the less it matters.

Distances are not similarities. L2 and dot-product scores need converting to "larger is better" before any of this. Vendors differ: Loams' Qdrant gateway documents that its DBSF over Euclid prefetches normalizes Loams' larger-is-better scores, while Qdrant normalizes raw distances.

What k in RRF doesCopy Anchor Link

The k constant decides how much rank 1 is worth compared with rank 20.

With k = 60, rank 1 is worth 1/61 and rank 20 is worth 1/80, about three quarters as much. RRF then behaves like a vote, and agreement between retrievers dominates. With a small k the top of each list dominates. A simple way to see the crossover: a document ranked first by one retriever and absent from the other scores 1/(k+1), and a document ranked r by both scores 2/(k+r). The solo winner outranks the consensus document once r > k + 2. At k = 60, a document has to be ranked worse than 62nd in both lists to lose to a solo #1. At k = 1, a document ranked fourth in both already loses.

Cormack et al. set k = 60 in a pilot investigation and didn't change it afterwards. Their pilot table shows MAP rising from 0.2072 at k = 0 to a plateau of about 0.214 from k = 30 to 90. They called 60 "near-optimal" and "not critical". Note that this was fusing 30 TREC-style runs, not two retrievers. With only two inputs, k matters more, as Bruch et al. found.

Defaults across vendors are not the same, and the difference is easy to miss:

Qdrant's default is the outlier. Its formula uses 0-based ranks with k = 2, so its top hit scores 1/2, the same as k = 1 with 1-based ranks. Moving a workload from Qdrant to Elasticsearch, or the other way, changes RRF's behaviour from top-heavy to vote-like with no change to your code.

Query-dependent weightingCopy Anchor Link

A single global α is a compromise. Keyword-shaped queries (SKUs, error codes, names) want BM25. Natural-language questions want vectors. Pinecone's docs suggest starting points of 0.75 for natural-language queries, 0.5 for mixed ones and 0.25 for keyword-heavy ones, on the vector side. You can make the weight a function of the query:

• Rules. Route by query shape: quoted strings, identifiers or one or two rare terms lean lexical. These are cheap and explainable.
• Retriever confidence. Use features of each list: the top score's margin over the next, the BM25 maximum, the number of lexical matches. A query with no strong BM25 hit gets a higher vector weight.
• A model. DAT (Hsu and Tzeng, 2025) asks an LLM to judge the top-1 result of each retriever and sets α per query from those judgments. It reports gains over the best fixed α on SQuAD and DRCD, at the cost of LLM calls on the query path.

Each of these needs labelled queries to show it helps, and a fusion layer that accepts per-request weights.

Evaluating a fusion choiceCopy Anchor Link

Fusion is a hyperparameter search over a few numbers, so the risk isn't compute. It's overfitting a small query set and measuring the wrong thing.

Datasets. BEIR (18 datasets, NeurIPS 2021 Datasets and Benchmarks track) is the standard zero-shot suite, and it is what Bruch et al. used. MTEB groups many retrieval sets under one leaderboard, scored by NDCG@10. Its main finding is that no embedding method dominates every task, which is itself an argument for hybrid. MS MARCO passage is the large in-domain set; its official metric is MRR@10, and its labels are sparse (about one relevant passage per query), so it rewards finding the answer rather than ranking several. Most importantly, build a set from your own traffic. Fifty to a few hundred real queries with graded judgments will tell you more about your corpus than any public benchmark.

Metrics. Measure two things at two depths:

• Recall@k for each retriever and for the fused list, at the depth you fuse or rerank (100, 1,000). This tells you whether the candidates are there at all. A reranker can't promote what fusion dropped.
• NDCG@10 of the final list. This is what users see. Report MRR@10 too if your task has one right answer.

Method. Split the queries into tuning and test sets before you look at anything. Sweep α (or k, or the weights) on the tuning split, then report once on the test split. Report per-dataset numbers, not just the mean, because a fusion that helps on average often hurts one domain badly, which is Bruch et al.'s RRF transfer result. Use a paired significance test over queries (a paired t-test or a randomization test) rather than eyeballing a 0.01 difference. ranx implements the metrics, the tests and most fusion functions, including RRF, CombSUM with several normalizations and a weight optimizer, which makes it a good harness for this. trec_eval is the reference implementation of the metrics.

Baselines. Always report BM25 alone and vectors alone next to the hybrid. A hybrid that beats neither is common on small or very clean corpora, and it is cheaper to find that out before shipping two indexes.

Production concernsCopy Anchor Link

Per-signal top-k is a recall knob, not a page size. Each retriever needs to return enough candidates for the fused list to be good. Elasticsearch's rank_window_size and Qdrant's prefetch limit exist for this. Qdrant requires each prefetch limit to be at least limit + offset, or a page can come back empty. Too small a window cuts recall and, with min-max or DBSF, changes the normalization. Too large a window costs latency. For an ANN index, it also means the index must find more true neighbours, which usually means more probes. Published examples range from 20 per signal (pgvector's RRF example) to several hundred. Size it by measuring recall at the window against exact search.

Latency. The retrievers are independent, so run them in parallel, which makes the fused latency the slowest retriever's, plus the merge. The merge itself is negligible: a hash join over a few hundred ids. A cross-encoder is not negligible. It does one forward pass per candidate, so it is usually limited to the top 20 to 100 fused candidates. Vespa's global phase defaults to reranking 100 hits, and Elasticsearch's text_similarity_reranker defaults to rank_window_size 10.

Filter before fusion, not after. If you filter the fused list, a selective filter (one tenant, one week) can remove most of each retriever's top k, and you return fewer than limit results, or none. Apply the filter inside each retriever. For BM25 this is free, since a filter is just another clause. For ANN it needs filtered vector search: brute force on a small allowed set, a filtered index search for selective filters, or over-fetch and post-filter for broad ones. Elasticsearch's rrf and linear retrievers take a top-level filter applied to every child retriever for this reason.

Pagination stability. Fused rankings are recomputed for every request, so page 2 is only consistent with page 1 if four things hold:

1. Each retriever returns the same candidates, so the per-signal window must not depend on the offset. OpenSearch's pagination_depth exists so that every page draws from the same candidate set. Its docs say to keep it constant across pages.
2. The data doesn't change between pages. Otherwise you need a snapshot or point-in-time read.
3. Ties are broken deterministically. RRF produces many exact ties (every pair of documents at mirrored ranks), and an unstable sort makes results reshuffle between pages.
4. The approximate vector search is deterministic for a given index state.

Deep pagination over a fused list is bounded by the window anyway. Past rank_window_size, there is nothing to page into.

How Loams does itCopy Anchor Link

Loams' query engine (loams-query) plans every read as an Apache DataFusion plan, whatever API it arrived on: the native REST and gRPC query, SQL and Flight SQL, the Elasticsearch _search surface or Qdrant's query. Hybrid fusion is one physical operator, FusionExec, shared by all of them. The facts below are from the dev branch; the design is design 05 §4, and the hybrid retrieval concept page explains the model.

Every Loams claim below carries its status from the 0.1.0 release notes and the design documents: Available ships in 0.1.0, Experimental ships but may change, and Planned is designed and approved but not built. A gap that no design covers yet says so in words.

Three fusion methods. Available FusionExec implements:

• rrf with a configurable k, default 60, as 1/(k + rank) with 1-based ranks.
• dbsf, a port of Qdrant's distr_norm: an f32 Welford mean and sample variance, (s − (μ − 3σ)) / 6σ, not clamped, and 0.5 for a list of one hit or of identical scores.
• weighted, a weighted sum of the retrievers' raw scores, with weight 1.0 for any retriever without a weight.

Every input list is ordered by (score descending, primary key ascending). Contributions are summed in f32 in input order, and the fused list is ordered by (fused score descending, primary key ascending). So ties, which RRF produces often, always break the same way.

One request, several retrievers. Available The native body (POST /v1/namespaces/{ns}/query) takes a retrieve list of vector and text retrievers, each with its own k, plus filter, fuse, limit and offset:

{
  "from": "collections.articles",
  "retrieve": [
    { "vector": { "field": "embedding", "query": [0.12, 0.08, -0.31], "k": 100 } },
    { "text": { "field": "body", "query": "durable execution on object storage", "k": 100 } }
  ],
  "filter": { "term": { "lang": "en" } },
  "fuse": { "method": "rrf", "k": 60 },
  "limit": 10
}

fuse is {"method": "rrf", "k": …}, {"method": "dbsf"} or {"method": "weighted", "weights": [0.7, 0.3]}. With several retrievers and no fuse, the engine uses RRF with k = 60. Underneath, the request IR also has sparse-vector retrievers, nested fused retrievers (fusion of fusions, up to four levels deep) and an exact vector rescore retriever. The Qdrant gateway uses these for prefetch trees. A request may have at most 16 retrievers, and every k, and offset + limit, is at most 100,000.

SQL. Available The same operator is reachable from SQL through table functions named after Spice's: rrf(vector_search(…), text_search(…) [, k [, limit]]) fuses two or more retrievers of one collection with k = 60 by default. hybrid_search(collection, vector_field, vector, text_field, text, k [, 'rrf' | 'dbsf' [, rrf_k [, filter_json]]]) is the two-field shorthand. Each function returns _id, _score and the collection's columns, so a fused result can be joined and filtered like any table.

Compatibility surfaces map onto it. Available The Elasticsearch gateway reads retriever.rrf and the legacy rank.rrf, with rank_constant (default 60, at least 1) and rank_window_size. The Qdrant gateway maps fusion: rrf and fusion: dbsf onto the same operator. Because Qdrant's RRF uses 0-based ranks, it sends Qdrant's k (default 2) as k − 1, so a Qdrant client gets Qdrant's scores.

Filters run inside each retriever. Available The request's filter becomes a FilterBitmapExec that restricts the vector, text and sparse retrievers before they rank. It is ANDed with a retriever's own filter if it has one. Fusion never sees a document that fails the filter, so a selective filter doesn't empty the page. For vector retrieval the planner picks a strategy by selectivity. A filter allowing at most max(1,000, full_scan_threshold_kb · 1024 / (4 · dim)) rows is scored by brute force, and one allowing at most 10% of rows prefilters the index search. A broader one post-filters, over-fetching k · 1.5 / selectivity candidates and retrying once with four times more (design 05 §3). Vector scores are always recomputed by Loams' exact kernel, so the score fusion sees doesn't depend on which index served the candidate.

The window and the page. Available The fused list is truncated to the largest retriever k, and that is the window. score_threshold then applies to the fused score, and offset and limit cut the page. Documents are fetched only for the page (late materialization). Every read resolves a snapshot (a manifest version plus log offsets) and can be pinned with a consistency token. Combined with deterministic tie-breaking and exact vector scores, that means page 2 comes from the same fused list as page 1, as long as you keep the per-retriever k constant and read at the same token.

A rerank stage, without a model. Experimental The native body also accepts a rerank stage after fusion (and after graph expand, which is what it was built for). rerank.vector rescores the final candidates by cosine similarity of a vector field to a query vector. rerank.inherit scores graph neighbours from their seeds. rerank.model, which would call an external reranker endpoint, is defined in the request format and always refused: "rerank.model calls an external reranker, which is off on this server". The SQL rerank(…) table function is registered only so that it can refuse with a clear message.

What Loams does not do yetCopy Anchor Link

These are the gaps, checked against the code rather than the roadmap. Most of them have no design yet:

• No per-signal ranks or scores in the response (not designed). A hit carries its primary key and one fused _score. To see why a document ranked where it did, run each retriever on its own. This is the first thing we would add for debugging and offline evaluation.
• weighted is unnormalized (no normalized option is designed). It multiplies raw scores, so BM25 and cosine are on their own scales (the first diagram). There is no min-max, theoretical min-max or z-score option, so a convex combination in the Bruch et al. sense isn't available as a built-in. DBSF is the normalized score-based option, with the small-list noise described above. If you want TM2C2, compute it client-side from the two single-retriever queries, or pick weights per query.
• No weighted RRF (not designed). rrf has one k and no per-retriever weights. The Qdrant gateway refuses Qdrant's weighted RRF as unsupported, and Elasticsearch's per-retriever RRF weights (9.2+) have no native equivalent.
• No model reranker and no embedding endpoint. The model reranker is Planned (design 05 §4 describes a pluggable RerankExec, off by default); an embedding endpoint is not designed. Loams runs no models. Vector queries must arrive as vectors, the Elasticsearch _infer endpoint answers 404, and Qdrant inference objects are refused. Cross-encoder reranking happens in your application, on the fused candidates.
• No query-dependent weighting built in (not designed). Weights and k are per request, so the application can choose them per query, but the engine has no routing or confidence features.
• Ranking expressions are designed, not built. Planned Design 05 describes a post-fusion rank expression (decay, recency, field boosts) that ES function_score and Qdrant formula would compile to. The native body doesn't accept a rank key yet.
• hybrid_search in SQL takes only rrf or dbsf, not weighted.

RecommendationsCopy Anchor Link

• With no labelled queries, use RRF with k = 60, a per-signal window of about 100, and the filter inside each retriever. Check what k your engine actually uses.
• Once you have even 50 to 100 labelled queries, try a convex combination over theoretically normalized scores, and tune α on a held-out split. Keep RRF if it isn't clearly worse; it is more robust to change.
• Spend your latency on a cross-encoder over the top 20 to 50 fused candidates before you spend it on fancier fusion. It is usually the bigger gain.
• Measure recall at the window and NDCG@10 at the page, per dataset, against BM25-only and vector-only baselines.

Appendix A: a live comparison on DBpediaCopy Anchor Link

We ran BM25-only, vector-only, RRF (k = 60) and two weighted fusions on a real Loams 0.1.0 engine. This is an illustration, not a benchmark. It is eight topics, judged by one person (the author), on one small corpus, with a vector query that is not a text embedding. Read it as "what these functions did on these eight queries", not as evidence that one method is better in general.

Setup. The collection holds 100,000 DBpedia entities from Qdrant's public set on Hugging Face, with title and body as BM25 text fields and a 512-dimension OpenAI text-embedding-3-small vector per entity (cosine). Loams runs no embedding model, and we didn't send the queries to OpenAI, so the vector side is query by example: its query is the stored embedding of a seed article for the topic. The text side is a few keywords for the same need. The seed is removed from every result list. Each retriever fetches its top 100, and we score the top 10. Every request reads at one consistency token, so the five result lists come from the same state, and a rerun at that token returns the same lists.

Methods.

1. bm25: the text retriever alone.
2. vector: the vector retriever alone.
3. rrf60: both retrievers, fused with RRF at k = 60.
4. weighted_naive: Loams' weighted fusion with weights 0.5 and 0.5.
5. weighted_scaled: weights 1.0 for vector and 0.05 for BM25.

Fusion 5 compensates for scale. We picked 0.05 before scoring anything, from the rough 20-to-1 ratio of BM25 to cosine scores on one query outside the set, and didn't tune it on the judgments. A later check found top BM25 scores of 10 to 22 across the eight topics and top cosines of 0.54 to 0.78.

Judgments. For each topic, we pooled every method's top 10 and judged each pooled article relevant or not from its title and first sentence, against the written need (for example "a railway line in Japan": stations don't count). The pools held 18 to 24 articles per topic. Topic jprivers has only one relevant article in its pool, so its NDCG swings between 0 and 1 on a single document.

Results. Each cell is precision@10 / NDCG@10 (binary gains):

What it shows, with the caveats above:

• Neither single retriever wins every topic. BM25 wins sax and bavaria, where the keywords say exactly what a relevant article says. The vector side wins jprail, rome and moons, where the seed's category is the need. BM25 for "railway line Japan" mostly returns stations, and for "ancient Roman baths temple Rome" it returns battles and people.
• RRF was never the worst method on a topic, and it always landed between the two single retrievers or above both. That is the robustness argument for it as a default. On langs and canals it beat both single retrievers, because each retriever found relevant articles the other missed.
• Naive weighting is roughly BM25. With weights 0.5 and 0.5, BM25's raw scores (about 6 to 22 over each top 100) swamp the cosines (about 0.38 to 0.78). Its precision matches bm25 on five topics and is within 0.1 on the other three, as the normalization section predicts.
• Scale-aware weighting had the best mean NDCG@10 (0.724 against RRF's 0.673). That fits Bruch et al.'s finding, but eight topics and one assessor can't separate the two, and the 0.05 weight only works because these queries' score ranges were similar. A query whose top BM25 score is 3 would need a different weight, which is why a normalized convex combination is on the gap list above.

The script, as run (it only reads; set LOAMS_URL to your engine):

"""BM25-only, vector-only, RRF(k=60) and weighted fusion on e2e/dbpedia.

Read-only: every call is POST /v1/namespaces/e2e/query. Query-by-example: the
vector query is the stored embedding of a seed article (Loams runs no
embedding model, and the collection's vectors are OpenAI's), and the text
query is a few keywords for the same information need. The seed itself is
dropped from every result list.

    python3 -I fusion_live.py pool      # print the pooled top 10s to judge
    python3 -I fusion_live.py score     # score against QRELS below
"""
import json
import math
import os
import sys
import urllib.request

ENGINE = os.environ.get("LOAMS_URL", "http://engine.example:8080")
K, LIMIT = 100, 10
W_TEXT = 0.05  # raw BM25 is ~20x cosine here; set from one query outside the set

TOPICS = [
    # (id, seed title, text query, information need: what a relevant article is)
    ("langs", "Delphi (programming language)", "programming language",
     "a programming language"),
    ("moons", "Narvi (moon)", "moon of Saturn", "a natural satellite of a planet"),
    ("sax", "Ravi Coltrane", "jazz saxophonist", "a jazz saxophonist"),
    ("canals", "Rhine–Main–Danube Canal", "canal waterway", "a canal"),
    ("jprivers", "Fuji River", "river Japan", "a river in Japan"),
    ("jprail", "Chichibu Main Line", "railway line Japan", "a railway line in Japan"),
    ("rome", "Baths of Trajan", "ancient Roman baths temple Rome",
     "an ancient Roman building or monument"),
    ("bavaria", "Kahl am Main", "municipality Bavaria Germany",
     "a municipality in Bavaria"),
]


def post(body):
    req = urllib.request.Request(
        f"{ENGINE}/v1/namespaces/e2e/query", json.dumps(body).encode(),
        {"content-type": "application/json"})
    with urllib.request.urlopen(req, timeout=60) as r:
        return json.load(r)


def seed(title):
    hits = post({"from": "collections.dbpedia",
                 "retrieve": [{"text": {"field": "title", "query": title, "k": 50}}],
                 "select": ["title", "_vectors.embedding"], "limit": 50})["hits"]
    hit = next(h for h in hits if h["source"]["title"] == title)
    return hit["pk"], hit["vectors"]["embedding"]


def run(vec, text, method, token):
    vector = {"vector": {"field": "embedding", "query": vec, "k": K}}
    bm25 = {"text": {"field": "body", "query": text, "k": K}}
    body = {"from": "collections.dbpedia", "consistency": {"token": token},
            "select": ["title", "body"], "limit": LIMIT + 1}
    if method == "bm25":
        body["retrieve"] = [bm25]
    elif method == "vector":
        body["retrieve"] = [vector]
    elif method == "rrf60":
        body["retrieve"], body["fuse"] = [vector, bm25], {"method": "rrf", "k": 60}
    elif method == "weighted_naive":
        body["retrieve"], body["fuse"] = [vector, bm25], {"method": "weighted", "weights": [0.5, 0.5]}
    elif method == "weighted_scaled":
        body["retrieve"], body["fuse"] = [vector, bm25], {"method": "weighted", "weights": [1.0, W_TEXT]}
    return post(body)["hits"]


METHODS = ["bm25", "vector", "rrf60", "weighted_naive", "weighted_scaled"]

# Pooled judgments (one assessor, binary, from title and first sentence):
# topic id -> the relevant primary keys among every method's top 10.
QRELS = {
    "langs": {13432, 15654, 29839, 38235, 55594, 59110, 64972, 86656},
    "moons": {344, 848, 21738, 25459, 43642, 54768, 70743, 82951, 87035},
    "sax": {17859, 23344, 37682, 42076, 48431, 57854, 59561, 64765, 70849, 76946, 99208},
    "canals": {5256, 20595, 21090, 25193, 49661, 52654, 54509, 65026, 77229, 89729},
    "jprivers": {7157},
    "jprail": {21757, 32729, 33205, 42203, 56038, 60999, 65277, 88032, 92251, 99738},
    "rome": {21351, 26488, 32470, 33461, 36831, 38447, 76540, 78776, 82514},
    "bavaria": {19704, 28991, 46473, 57416, 58369, 60523, 72413, 72904, 72962, 77042,
                78941, 84583, 92505},
}


def results():
    req = urllib.request.Request(
        f"{ENGINE}/v1/namespaces/e2e/collections/dbpedia/documents/count", b"{}",
        {"content-type": "application/json"})
    with urllib.request.urlopen(req, timeout=60) as r:
        token = json.load(r)["read_token"]
    out = {}
    for tid, title, text, _need in TOPICS:
        pk, vec = seed(title)
        out[tid] = {}
        for m in METHODS:
            hits = [h for h in run(vec, text, m, token) if h["pk"] != pk][:LIMIT]
            out[tid][m] = [(h["pk"], h["source"]["title"], h["source"].get("body", "")) for h in hits]
    return token, out


def ndcg(ranked, rel):
    dcg = sum(1 / math.log2(i + 2) for i, pk in enumerate(ranked) if pk in rel)
    ideal = sum(1 / math.log2(i + 2) for i in range(min(len(rel), LIMIT)))
    return dcg / ideal if ideal else 0.0


if __name__ == "__main__":
    token, out = results()
    print("read token", token)
    if sys.argv[1:] == ["pool"]:
        for tid, title, text, need in TOPICS:
            pool = {}
            for m in METHODS:
                for pk, t, b in out[tid][m]:
                    pool.setdefault(pk, (t, b))
            print(f"\n## {tid}: {need}  (seed {title!r}, text {text!r}, pool {len(pool)})")
            for pk, (t, b) in sorted(pool.items()):
                print(f"{pk}\t{t}\t{b[:160]!r}")
    else:
        print(f"{'topic':10s}" + "".join(f"{m:>17s}" for m in METHODS))
        sums = {m: [0.0, 0.0] for m in METHODS}
        for tid, *_ in TOPICS:
            rel = QRELS[tid]
            row = f"{tid:10s}"
            for m in METHODS:
                ranked = [pk for pk, _, _ in out[tid][m]]
                p, n = sum(pk in rel for pk in ranked) / LIMIT, ndcg(ranked, rel)
                sums[m][0] += p
                sums[m][1] += n
                row += f"{p:9.1f} / {n:5.3f}"
            print(row)
        t = len(TOPICS)
        print(f"{'mean':10s}" + "".join(f"{s[0] / t:9.2f} / {s[1] / t:5.3f}" for s in sums.values()))

python3 fusion_live.py pool prints every method's top 10 for judging, and python3 fusion_live.py score prints the table. Running it against another engine or another load of the collection gives different primary keys, so you have to judge your own pools again.

Appendix B: the worked exampleCopy Anchor Link

The numbers in the first diagram and the claims about missing documents and k come from this script. It is a hand-made illustration of how the functions behave, not a benchmark: five BM25 hits and five vector hits for one imaginary query, with scales typical of BM25 and of a sentence-embedding model. It needs Python 3 and nothing else.

from statistics import mean, stdev

bm25 = {"d1": 21.4, "d2": 12.9, "d3": 11.7, "d4": 6.2, "d5": 5.8}
cosine = {"d6": 0.842, "d3": 0.836, "d7": 0.829, "d2": 0.811, "d8": 0.804}
docs = sorted(set(bm25) | set(cosine))

def ranks(scores):
    order = sorted(scores, key=lambda d: (-scores[d], d))
    return {d: i + 1 for i, d in enumerate(order)}

def minmax(scores, lo=None):
    lo = min(scores.values()) if lo is None else lo
    hi = max(scores.values())
    return {d: (s - lo) / (hi - lo) for d, s in scores.items()}

def dbsf(scores):  # Qdrant's distr_norm: mu +- 3 sigma, not clipped
    mu, sd = mean(scores.values()), stdev(scores.values())
    return {d: (s - (mu - 3 * sd)) / (6 * sd) for d, s in scores.items()}

def fuse(parts, weights):
    out = {d: sum(w * p.get(d, 0.0) for p, w in zip(parts, weights)) for d in docs}
    return sorted(out.items(), key=lambda kv: (-kv[1], kv[0]))

def rrf(k):
    parts = [{d: 1 / (k + r) for d, r in ranks(s).items()} for s in (bm25, cosine)]
    return fuse(parts, [1, 1])

methods = {
    "raw 0.5/0.5": fuse([bm25, cosine], [0.5, 0.5]),
    "min-max 0.5/0.5": fuse([minmax(bm25), minmax(cosine)], [0.5, 0.5]),
    "TM2C2 a=0.5": fuse([minmax(bm25, 0.0), minmax(cosine, -1.0)], [0.5, 0.5]),
    "DBSF": fuse([dbsf(bm25), dbsf(cosine)], [1, 1]),
    "RRF k=60": rrf(60),
    "RRF k=1": rrf(1),
}
for name, ranking in methods.items():
    print(f"{name:16s} " + "  ".join(f"{d}:{s:.4f}" for d, s in ranking[:5]))

Its output (top five per method):

raw 0.5/0.5      d1:10.7000  d2:6.8555  d3:6.2680  d4:3.1000  d5:2.9000
min-max 0.5/0.5  d3:0.6102  d1:0.5000  d6:0.5000  d7:0.3289  d2:0.3197
TM2C2 a=0.5      d2:0.7930  d3:0.7717  d1:0.5000  d6:0.5000  d7:0.4965
DBSF             d3:1.1213  d2:0.8971  d1:0.7578  d6:0.6801  d7:0.5471
RRF k=60         d3:0.0320  d2:0.0318  d1:0.0164  d6:0.0164  d7:0.0159
RRF k=1          d3:0.5833  d2:0.5333  d1:0.5000  d6:0.5000  d7:0.2500

What to read from it:

• Raw sums are BM25's ranking. The vector retriever's best hit, d6, isn't in the top five. Loams' weighted method computes exactly this line, so its weights have to absorb the scale difference.
• Min-max pushes d2 (BM25 second, vector fourth) down to fifth, because its good cosine of 0.811 normalizes to 0.18.
• TM2C2 gives the five vector hits nearly identical normalized scores (0.98 to 1.0), so BM25 decides the order. That is why α has to lean toward the semantic side.
• DBSF and both RRFs put the two documents both retrievers found (d3, d2) on top, which is the behaviour most people want from hybrid search.
• RRF at k = 60 produces near-ties (0.0164 against 0.0164 against 0.0159) that only a deterministic tie-break keeps stable across requests.