Two rankings, one score
Each row is a depth d. RBO compares the top-d prefixes of both rankings and takes their agreement Ad, the fraction of items they share. It then weights that agreement by wd. Lines connect shared items; a line's thickness shows how much that item adds to the score at the current p. Drag items to reorder them.
Best item first. Separate items with spaces, commas or new lines (use commas or new lines if items contain spaces). Up to 50 items per list. RBO assumes no ties, so repeated items are dropped.
- Lower bound RBOmin
- –
- Upper bound RBOmax
- –
- Residual (max − min)
- –
Data table
Where the weight goes
An item at rank r is part of every prefix from depth r on, and adds 1/d to the agreement at each such depth. Its total weight is therefore ωr = Σd ≥ r wd/d. The chart adds these up: the share of the total weight held by the top d ranks. A larger p pushes the curve to the right, so the score depends on deeper ranks, which short lists may not even contain.
Data table
At p = 0.9
Which pair is more similar? It depends on p
Six variations of the ranking S = A B C D E F G H I J, each compared with S. With a small p, RBO rewards agreement at the very top. As p grows, sharing many items counts more, almost regardless of their order. Drag p and watch the ranking of the pairs change. Select a pair to load it into the table above.
Data table
Most similar first, at p = 0.9
Each row shows ranking T; S is always A B C D E F G H I J. The colour key matches the chart.
The formulas behind the page
Definitions follow Webber, Moffat & Zobel (2010), who introduced RBO. All values on this page are computed exactly in your browser.
Agreement and depth weights
Each depth gets p times the weight of the depth before it. At p = 0.9, depth 10 still gets 39% of the weight of depth 1. At p = 0.5 it gets 0.2%. At p = 0, RBO only checks whether the two top items are the same.
Why p is called persistence
Imagine a reader who compares the rankings one depth at a time and, after each depth, carries on with probability p. The chance of stopping at depth d is exactly wd, so RBO is the expected agreement at the depth where the reader stops. On average that depth is
p = 0.9 models a reader who inspects about 10 items, and p = 0.98 one who inspects about 50.
How much a single rank counts
Summed over the top d ranks, this gives the share of the weight plotted above:
Rule of thumb: the top 1/(1 − p) ranks hold about 85% of the weight (85.6% at p = 0.9, 85.2% at p = 0.98).
Finite lists: bounds and an estimate
Real rankings end at some depth k, so RBO can only be bounded. For two lists of length k, with no further overlap beyond depth k:
RBOmax = RBOmin + RBOres assumes that every unseen item matches as early as possible. The point estimate assumes that the agreement at depth k continues forever:
The depths beyond k carry pk of the weight, so the gap between the bounds grows quickly with p. At k = 10 that is 35% of the weight for p = 0.9, and 90% for p = 0.99. For lists of different lengths the page uses the paper's extension: past the end of the shorter list, its unseen items are assumed to match at the rate seen so far.
Choosing p in practice
- Pick p so that 1/(1 − p) matches the depth that matters. Use p = 0.9 for a top-10 result page, or p = 0.98 if agreement down to about rank 50 should count.
- Make the lists long enough for that p. Close to p = 1, short lists leave a large residual, so report the bounds along with RBOext.
- Always report p. RBO values computed with different p cannot be compared.