Base rate fallacy

The base rate fallacy, also called base rate neglect or base rate bias, is an error that occurs when the conditional probability of some hypothesis H given some evidence E is assessed without taking into account the "base rate" or "prior probability" of H and the total probability of evidence E.^{[1]}
Contents
Example
In a city of 1 million inhabitants there are 100 known terrorists and 999,900 nonterrorists. The base rate probability of one random inhabitant of the city being a terrorist is thus 0.0001 and the base rate probability of a random inhabitant being a nonterrorist is 0.9999. In an attempt to catch the terrorists, the city installs a surveillance camera with automatic facial recognition software. The software has two failure rates of 1%:
 if the camera sees a terrorist, it will ring a bell 99% of the time, and mistakenly fail to ring it 1% of the time (in other words, the falsenegative rate is 1%).
 if the camera sees a nonterrorist, it will not ring the bell 99% of the time, but it will mistakenly ring it 1% of the time (the falsepositive rate is 1%).
So, the failure rate of the camera is always 1%.
Suppose somebody triggers the alarm. What is the chance they are a terrorist?
Someone making the 'base rate fallacy' would incorrectly claim that there is a 99% chance that they are a terrorist, because 'the' failure rate of the camera is always 1%. Although it seems to make sense, it is actually bad reasoning. The calculation below will show that the chances they are a terrorist are actually near 1%, not near 99%.
The fallacy arises from confusing two different failure rates. The 'number of nonterrorists per 100 bells' and the 'number of nonbells per 100 terrorists' are unrelated quantities, and there is no reason one should equal the other. They don't even have to be roughly equal.
To show that they do not have to be equal, consider a camera that, when it sees a terrorist, rings a bell 20% of the time and fails to do so 80% of the time, while when it sees a nonterrorist, it works perfectly and never rings the bell. If this second camera rings, the chance that it failed by ringing at a nonterrorist is 0%. However if it sees a terrorist, the chance that it fails to ring is 80%. So, here 'nonterrorists per bell' is 0% but 'nonbells per terrorist' is 80%.
Now let's go back to our original camera, the one with 'bells per nonterrorist' of 1% and 'nonbells per terrorist' of 1%, and let's compute the 'nonterrorists per bell' rate.
Imagine that the city's entire population of one million people pass in front of the camera. About 99 of the 100 terrorists will trigger the alarm—and so will about 9,999 of the 999,900 nonterrorists. Therefore, about 10,098 people will trigger the alarm, among which about 99 will be terrorists. So the probability that a person triggering the alarm is actually a terrorist is only about 99 in 10,098, which is less than 1%, and very very far below our initial guess of 99%.
The base rate fallacy is only fallacious in this example because there are more nonterrorists than terrorists. If the city had about as many terrorists as nonterrorists, and the falsepositive rate and the falsenegative rate were nearly equal, then the probability of misidentification would be about the same as the falsepositive rate of the device. These special conditions hold sometimes: as for instance, about half the women undergoing a pregnancy test are actually pregnant, and some pregnancy tests give about the same rates of false positives and of false negatives. In this case, the rate of false positives per positive test will be nearly equal to the rate of false positives per nonpregnant woman. This is why it is very easy to fall into this fallacy: it gives the correct answer in many common situations.
In many realworld situations, though, particularly problems like detecting criminals in a largely lawabiding population, the small proportion of targets in the large population makes the base rate fallacy very applicable. Even a very low falsepositive rate will result in so many false alarms as to make such a system useless in practice.
Mathematical formalism
In the above example, where P(AB) means the probability of A given B, the base rate fallacy is the incorrect assumption that:
However, the correct expression uses Bayes' theorem to take into account the probabilities of both A and B, and is written as:
Thus, in the example the probability is overestimated by more than 100 times, due to the failure to take into account the fact that there are about 10000 times more nonterrorists than terrorists (a.k.a. failure to take into account the 'prior probability' of being a terrorist).
Findings in psychology
In some experiments, students were asked to estimate the grade point averages (GPAs) of hypothetical students. When given relevant statistics about GPA distribution, students tended to ignore them if given descriptive information about the particular student, even if the new descriptive information was obviously of little or no relevance to school performance.^{[citation needed]} This finding has been used to argue that interviews are an unnecessary part of the college admissions process because interviewers are unable to pick successful candidates better than basic statistics.^{[who?]}
Psychologists Daniel Kahneman and Amos Tversky attempted to explain this finding in terms of the representativeness heuristic. Richard Nisbett has argued that some attributional biases like the fundamental attribution error are instances of the base rate fallacy: people underutilize "consensus information" (the "base rate") about how others behaved in similar situations and instead prefer simpler dispositional attributions.
See also
 Bayesian probability
 Data dredging
 False positive paradox
 Inductive argument
 Misleading vividness
 Prosecutor's fallacy
Notes
References
 BarHillel, M. (1980). The baserate fallacy in probability judgments. Acta Psychologica, 44, 211233.
 Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80, 237251. (summary here)
 Nisbett, R.E., Borgida, E., Crandall, R., & Reed, H. (1976). Popular induction: Information is not always informative. In J.S. Carroll & J.W. Payne (Eds.), Cognition and social behavior, 2, 227236.
External links
 The Base Rate Fallacy The Fallacy Files
 Psychology of Intelligence Analysis: Base Rate Fallacy
Categories: Relevance fallacies
 Cognitive biases
 Behavioral finance
Wikimedia Foundation. 2010.
Look at other dictionaries:
base rate fallacy — noun A common error in logical reasoning where an effect is attributed to an incorrect cause because the basic statistical ratios have not been taken into account … Wiktionary
Fallacy — In logic and rhetoric, a fallacy is usually incorrect argumentation in reasoning resulting in a misconception or presumption. By accident or design, fallacies may exploit emotional triggers in the listener or interlocutor (appeal to emotion), or… … Wikipedia
Prosecutor's fallacy — The prosecutor s fallacy is a fallacy of statistical reasoning made in law where the context in which the accused has been brought to court is falsely assumed to be irrelevant to judging how confident a jury can be in evidence against them with a … Wikipedia
Naturalistic fallacy — The naturalistic fallacy is often claimed to be a formal fallacy. It was described and named by British philosopher G. E. Moore in his 1903 book Principia Ethica. Moore stated that a naturalistic fallacy is committed whenever a philosopher… … Wikipedia
Deductive fallacy — A deductive fallacy is defined as a deductive argument that is invalid. The argument itself could have true premises, but still have a false conclusion.[1] Thus, a deductive fallacy is a fallacy where deduction goes wrong, and is no longer a… … Wikipedia
Moralistic fallacy — The moralistic fallacy is in essence the reverse of the naturalistic fallacy. Naturalistic fallacy presumes that what is or what occurs forms what ought to be. Thus the observed natural is reasoned a priori as moral.[1] Moralistic fallacy implies … Wikipedia
Genetic fallacy — The genetic fallacy is a fallacy of irrelevance where a conclusion is suggested based solely on something or someone s origin rather than its current meaning or context. This overlooks any difference to be found in the present situation,… … Wikipedia
Perfect solution fallacy — The perfect solution fallacy is a logical fallacy that occurs when an argument assumes that a perfect solution exists and/or that a solution should be rejected because some part of the problem would still exist after it was implemented.Presumably … Wikipedia
List of fallacies — For specific popular misconceptions, see List of common misconceptions. A fallacy is incorrect argumentation in logic and rhetoric resulting in a lack of validity, or more generally, a lack of soundness. Contents 1 Formal fallacies 1.1… … Wikipedia
List of cognitive biases — A cognitive bias is a pattern of poor judgment, often triggered by a particular situation. Identifying poor judgment, or more precisely, a deviation in judgment, requires a standard for comparison, i.e. good judgment . In scientific… … Wikipedia