Precision and recall

Precision and Recall are two widely used measures for evaluating the quality of results in domains such as Information Retrieval and statistical classification.

Precision can be seen as a measure of exactness or fidelity, whereas Recall is a measure of completeness.

In an Information Retrieval scenario, Precision is defined as the "number of relevant documents" retrieved by a search "divided by the total number of documents retrieved" by that search, and Recall is defined as the "number of relevant documents" retrieved by a search "divided by the total number of existing relevant documents" (which should have been retrieved).

In a statistical classification task, the Precision for a class is the "number of true positives" (i.e. the "number of items correctly labeled as belonging to the class") "divided by the total number of elements labeled as belonging to the class" (i.e. the sum of true positives and false positives, which are items incorrectly labeled as belonging to the class). Recall in this context is defined as the "number of true positives" "divided by the total number of elements that actually belong to the class" (i.e. the sum of true positives and false negatives, which are items which were not labeled as belonging to that class but should have been).

In Information Retrieval, a perfect Precision score of 1.0 means that every result retrieved by a search was relevant (but says nothing about whether all relevant documents were retrieved) whereas a perfect Recall score of 1.0 means that all relevant documents were retrieved by the search (but says nothing about how many irrelevant documents were also retrieved).

In a classification task, a Precision score of 1.0 for a class C means that every item labeled as belonging to class C does indeed belong to class C (but says nothing about the number of items from class C that were not labeled correctly) whereas a Recall of 1.0 means that every item from class C was labeled as belonging to class C (but says nothing about how many other items were incorrectly also labeled as belonging to class C).

Often, there is an inverse relationship between Precision and Recall, where it is possible to increase one at the cost of reducing the other. For example, an information retrieval system (such as a search engine) can often increase its Recall by retrieving more documents, at the cost of increasing number of irrelevant documents retrieved (decreasing Precision).Similarly, a classification system for deciding whether or not, say, a fruit is an orange, can achieve high Precision by only classifying fruits with the exact right shape and color as oranges, but at the cost of low Recall due to the number of "false negatives" from oranges that did not quite match the specification.

Usually, Precision and Recall scores are not discussed in isolation. Instead, either values for one measure are compared for a fixed level at the other measure (e.g. "precision at a recall level of 0.75") or both are combined into a single measure, such as the F-measure, which is the "weighted harmonic mean of precision and recall" (see below).

Definition(Information Retrieval context)

In Information Retrieval contexts, Precision and Recall are defined in terms of a set of retrieved documents (e.g. the list of documents produced by a web search engine for a query) and a set of relevant documents (e.g. the list of all documents on the internet that are relevant for a certain topic).

mbox{Recall}=frac

Definition (classification context)

In the context of classification tasks, the terms true positives, true negatives, false positives and false negatives (see also Type I and type II errors) are used to compare the given classification of an item (the class label assigned to the item by a classifier) with the desired correct classification (the class the item actually belongs to). This is illustrated by the table below:

Precision and Recall are then defined as

mbox{Recall}=frac{tp}{tp+fn}

mbox{Precision}=frac{tp}{tp+fp}

Probabilistic Interpretation

It is possible to interpret Precision and Recall not as ratios but as probabilities:

* Recall is the probability that a (randomly selected) relevant document is retrieved in a search.

* Precision is the probability that a (randomly selected) retrieved document is relevant.

F-measure

A popular measure that combines Precision and Recall is the weighted harmonic mean of precision and recall, the traditional F-measure or balanced F-score:

:F = 2 cdot (mathrm{precision} cdot mathrm{recall}) / (mathrm{precision} + mathrm{recall}).,

This is also known as the F_1 measure, because recall and precision are evenly weighted.

It is a special case of the general F_eta measure (for non-negative real values of eta):

:F_eta = (1 + eta^2) cdot (mathrm{precision} cdot mathrm{recall}) / (eta^2 cdot mathrm{precision} + mathrm{recall}).,

Two other commonly used F measures are the F_{2} measure, which weights recall twice as much as precision, and the F_{0.5} measure, which weights precision twice as much as recall.

The F-measure was derived by van Rijsbergen (1979) so that F_eta "measures the effectiveness of retrieval with respect to a user who attaches β times as much importance to recall as precision". It is based on van Rijsbergen's effectiveness measure E = 1-(1/(alpha/P + (1-alpha)/R)). Their relationship is F_eta = 1 - E where alpha=1/(eta^2+1).

ee also

* Information retrieval
* Binary classification

Sources

* Makhoul, John; Francis Kubala; Richard Schwartz; Ralph Weischedel: [http://citeseer.ist.psu.edu/makhoul99performance.html "Performance measures for information extraction".] In: "Proceedings of DARPA Broadcast News Workshop, Herndon, VA, February 1999".

* Baeza-Yates, R.; Ribeiro-Neto, B. (1999). "Modern Information Retrieval". New York: ACM Press, Addison-Wesley. Seiten 75 ff. ISBN 0-201-39829-X

* van Rijsbergen, C.V.: "Information Retrieval". London; Boston. Butterworth, 2nd Edition 1979. ISBN 0-408-70929-4

External links

* [http://www.dcs.gla.ac.uk/Keith/Preface.html Information Retrieval – C. J. van Rijsbergen 1979]


Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Precision und Recall — Die Artikel Positiver Vorhersagewert und Recall und Precision überschneiden sich thematisch. Hilf mit, die Artikel besser voneinander abzugrenzen oder zu vereinigen. Beteilige dich dazu an der Diskussion über diese Überschneidungen. Bitte… …   Deutsch Wikipedia

  • Recall und Precision — Die Artikel Positiver Vorhersagewert und Recall und Precision überschneiden sich thematisch. Hilf mit, die Artikel besser voneinander abzugrenzen oder zu vereinigen. Beteilige dich dazu an der Diskussion über diese Überschneidungen. Bitte… …   Deutsch Wikipedia

  • Recall — may refer to:*Product recall *Recall election *Letter to recall sent to return an ambassador from a country, either as a diplomatic protest or because the diplomat is being reassigned elsewhere and is being replaced by another envoy *Recall to… …   Wikipedia

  • Precision — steht für: William Beardmore and Company, ein ehemaliger Motorenhersteller Recall und Precision, ist ein Maß zur Beschreibung der Güte eines Suchergebnisses in der Informatik und in der Dokumentationswissenschaft Diese Seite ist eine …   Deutsch Wikipedia

  • Precision (information retrieval) — In the field of information retrieval, precision is the percent of retrieved documents that are relevant to the search:: mbox{precision}=frac{|{mbox{retrieved documents}} Precision takes all retrieved documents into account, but it can also be… …   Wikipedia

  • Sensitivity and specificity — are statistical measures of the performance of a binary classification test. The sensitivity or the recall rate measures the proportion of actual positives which are correctly identified as such (i.e. the percentage of sick people who are… …   Wikipedia

  • Accuracy and precision — In the fields of science, engineering, industry and statistics, accuracy is the degree of closeness of a measured or calculated quantity to its actual (true) value. Accuracy is closely related to precision, also called reproducibility or… …   Wikipedia

  • Computers and Information Systems — ▪ 2009 Introduction Smartphone: The New Computer.       The market for the smartphone in reality a handheld computer for Web browsing, e mail, music, and video that was integrated with a cellular telephone continued to grow in 2008. According to… …   Universalium

  • Tradition and Living Magisterium — • The word tradition refers sometimes to the thing (doctrine, account, or custom) transmitted from one generation to another sometimes to the organ or mode of the transmission Catholic Encyclopedia. Kevin Knight. 2006. Tradition and Living… …   Catholic encyclopedia

  • Powers and abilities of Superman — The powers of DC Comics character Superman have changed a great deal since his introduction in the 1930s. The extent of his powers peaked during the 1970s and 1980s to the point where various writers found it difficult to create suitable… …   Wikipedia


Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”

We are using cookies for the best presentation of our site. Continuing to use this site, you agree with this.