Interactive explainer · comparing rankings

Rank-biased overlap and its persistence parameter p

Rank-biased overlap (RBO) scores the similarity of two rankings by comparing their top-d prefixes at every depth d and averaging the overlap with geometrically decreasing weights. A single parameter, the persistence p, sets how fast those weights decay. It decides which ranks the score actually depends on. Move p and every number on this page updates.

RBO(S,T,p) = (1−p) ∑d=1∞ pd−1 Ad
Ad = |S:d∩T:d| d

S:d is the set of the top d items of ranking S. The depth weights wd = (1 − p)pd−1 sum to 1, so RBO runs from 0 (no shared items) to 1 (identical rankings).

Between 0 and 0.999. Arrow keys change the value.

Two rankings, one score

Each row is a depth d. RBO compares the top-d prefixes of both rankings and takes their agreement Ad, the fraction of items they share. It then weights that agreement by wd. Lines connect shared items; a line's thickness shows how much that item adds to the score at the current p. Drag items to reorder them.

RBOext, the point estimate, at p = 0.9
–
Lower bound RBOmin
–
Upper bound RBOmax
–
Residual (max − min)
–
00.51
guaranteed (≤ RBOmin) undetermined (depends on unseen items) estimate RBOext


This pair's RBO for every p Line: RBOext. Band: from RBOmin to RBOmax. Click or drag on the chart to set p.
Data table

Where the weight goes

An item at rank r is part of every prefix from depth r on, and adds 1/d to the agreement at each such depth. Its total weight is therefore ωr = Σd ≥ r wd/d. The chart adds these up: the share of the total weight held by the top d ranks. A larger p pushes the curve to the right, so the score depends on deeper ranks, which short lists may not even contain.

Share of the total weight held by the top d ranks Logarithmic depth axis. Grey curves are fixed reference values of p.
Data table

At p = 0.9

Which pair is more similar? It depends on p

Six variations of the ranking S = A B C D E F G H I J, each compared with S. With a small p, RBO rewards agreement at the very top. As p grows, sharing many items counts more, almost regardless of their order. Drag p and watch the ranking of the pairs change. Select a pair to load it into the table above.

RBOext of each pair as a function of p Hover for values; click or drag to set p.
Data table

Most similar first, at p = 0.9

    Each row shows ranking T; S is always A B C D E F G H I J. The colour key matches the chart.

    The formulas behind the page

    Definitions follow Webber, Moffat & Zobel (2010), who introduced RBO. All values on this page are computed exactly in your browser.

    Agreement and depth weights

    Xd= |S:d∩T:d| , Ad=Xdd , wd=(1−p)pd−1

    Each depth gets p times the weight of the depth before it. At p = 0.9, depth 10 still gets 39% of the weight of depth 1. At p = 0.5 it gets 0.2%. At p = 0, RBO only checks whether the two top items are the same.

    Why p is called persistence

    Imagine a reader who compares the rankings one depth at a time and, after each depth, carries on with probability p. The chance of stopping at depth d is exactly wd, so RBO is the expected agreement at the depth where the reader stops. On average that depth is

    𝔼[stopping depth]= ∑d=1∞d⁢wd =11−p

    p = 0.9 models a reader who inspects about 10 items, and p = 0.98 one who inspects about 50.

    How much a single rank counts

    ωr= ∑d=r∞ wdd = 1−pp ( ln⁡11−p − ∑i=1r−1 pii )

    Summed over the top d ranks, this gives the share of the weight plotted above:

    W(1:d)= 1−pd−1+ 1−pp⋅d⋅ (ln⁡11−p− ∑i=1d−1 pii)

    Rule of thumb: the top 1/(1 − p) ranks hold about 85% of the weight (85.6% at p = 0.9, 85.2% at p = 0.98).

    Finite lists: bounds and an estimate

    Real rankings end at some depth k, so RBO can only be bounded. For two lists of length k, with no further overlap beyond depth k:

    RBOmin= 1−pp ( ∑d=1k (Xd−Xk) pdd −Xkln⁡(1−p) )

    RBOmax = RBOmin + RBOres assumes that every unseen item matches as early as possible. The point estimate assumes that the agreement at depth k continues forever:

    RBOext= Xkkpk+ 1−pp ∑d=1k Xddpd

    The depths beyond k carry pk of the weight, so the gap between the bounds grows quickly with p. At k = 10 that is 35% of the weight for p = 0.9, and 90% for p = 0.99. For lists of different lengths the page uses the paper's extension: past the end of the shorter list, its unseen items are assumed to match at the rate seen so far.

    Choosing p in practice

    • Pick p so that 1/(1 − p) matches the depth that matters. Use p = 0.9 for a top-10 result page, or p = 0.98 if agreement down to about rank 50 should count.
    • Make the lists long enough for that p. Close to p = 1, short lists leave a large residual, so report the bounds along with RBOext.
    • Always report p. RBO values computed with different p cannot be compared.