Mean reciprocal rank


Mean reciprocal rank

Mean reciprocal rank is a statistic for evaluating any process that produces a list of possible responses to a query, ordered by probability of correctness. The reciprocal rank of a query response is the multiplicative inverse of the rank of the first correct answer. The mean reciprocal rank is the average of the reciprocal ranks of results for a sample of queries Q[1]:

 \text{MRR} = \frac{1}{|Q|} \sum_{i=1}^{|Q|} \frac{1}{\text{rank}_i}. \!

The reciprocal value of the mean reciprocal rank corresponds to the harmonic mean of the ranks.

Example

For example, suppose we have the following three sample queries for a system that tries to translate English words to their plurals. In each case, the system makes three guesses, with the first one being the one it thinks is most likely correct:

Query Results Correct response Rank Reciprocal rank
cat catten, cati, cats cats 3 1/3
torus torii, tori, toruses tori 2 1/2
virus viruses, virii, viri viruses 1 1

Given those three samples, we could calculate the mean reciprocal rank as (1/3 + 1/2 + 1)/3 = 11/18 or about 0.61.

This basic definition does not specify what to do if...

  1. none of the proposed results are correct (use mean reciprocal rank 0), or if
  2. there are multiple correct answers in the list. Consider using mean average precision (MAP).

See also Information retrieval and Question answering.[2]

References

  1. ^ E.M. Voorhees (1999). "Proceedings of the 8th Text Retrieval Conference". TREC-8 Question Answering Track Report. pp. 77–82. 
  2. ^ D. R. Radev, H. Qi, H. Wu, W. Fan (2002). "Evaluating web-based question answering systems". Proceedings of LREC. 

Wikimedia Foundation. 2010.

Look at other dictionaries:

  • MRR — Mean Reciprocal Rank (Academic & Science » Mathematics) *** Mid Atlantic Realty Trust (Business » NYSE Symbols) *** Minimum Risk Route (Governmental » Military) ** Motorized Rifle Regiment (Governmental » Military) * Market Research Report… …   Abbreviations dictionary

  • List of statistics topics — Please add any Wikipedia articles related to statistics that are not already on this list.The Related changes link in the margin of this page (below search) leads to a list of the most recent changes to the articles listed below. To see the most… …   Wikipedia

  • Обучение ранжированию — (англ. learning to rank или machine learned ranking, MLR)[1]  это класс задач машинного обучения с учителем, заключающихся в автоматическом подборе ранжирующей модели по обучающей выборке, состоящей из множества списков и заданных… …   Википедия

  • Information retrieval — This article is about information retrieval in general. For the fictional government department, see Brazil (film). Information retrieval (IR) is the area of study concerned with searching for documents, for information within documents, and for… …   Wikipedia

  • List of mathematics articles (M) — NOTOC M M estimator M group M matrix M separation M set M. C. Escher s legacy M. Riesz extension theorem M/M/1 model Maass wave form Mac Lane s planarity criterion Macaulay brackets Macbeath surface MacCormack method Macdonald polynomial Machin… …   Wikipedia

  • MRR — is an acronym that can stand for:In science: * Mean reciprocal rank, a search quality measure in information retrieval *Modulating retro reflector, Free space optical communications technologyIn Online Marketing: * Master Resell RightRoads called …   Wikipedia

  • Systèmes de questions-réponses — Les Systèmes de réponse à des questions explorent de nouvelles méthodes de recherche d information exploitant des requêtes formulées à l aide du langage naturel et non plus en se basant uniquement sur des mots clés (comme c est le cas avec les… …   Wikipédia en Français

  • List of mathematics articles (R) — NOTOC R R. A. Fisher Lectureship Rabdology Rabin automaton Rabin signature algorithm Rabinovich Fabrikant equations Rabinowitsch trick Racah polynomials Racah W coefficient Racetrack (game) Racks and quandles Radar chart Rademacher complexity… …   Wikipedia

  • Europe, history of — Introduction       history of European peoples and cultures from prehistoric times to the present. Europe is a more ambiguous term than most geographic expressions. Its etymology is doubtful, as is the physical extent of the area it designates.… …   Universalium

  • Germany — /jerr meuh nee/, n. a republic in central Europe: after World War II divided into four zones, British, French, U.S., and Soviet, and in 1949 into East Germany and West Germany; East and West Germany were reunited in 1990. 84,068,216; 137,852 sq.… …   Universalium


Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”

We are using cookies for the best presentation of our site. Continuing to use this site, you agree with this.