What is Base Rate Fallacy?

Base rate fallacy is a type of error that occurs when relevant data or commonly-understood (statistically relevant) information about a subject matter (base rate) is neglected or ignored in favor of new information.

Back to: Management & Organizational Behavior

Academics research on Base Rate Fallacy

• The base-rate fallacy in probability judgments, Bar-Hillel, M. (1980). The base-rate fallacy in probability judgments.Acta Psychologica,44(3), 211-233. The base-rate fallacy is people's tendency to ignore base rates in favor of,e.g., individuating information (when such is available), rather than integrate the two. This tendency has important implications for understanding judgment phenomena in many clinical, legal, and social-psychological settings. An explanation of this phenomenon is offered, according to which people order information by its perceived degree of relevance, and let high-relevance information dominate low-relevance information. Information is deemed more relevant when it relates more specifically to a judged target case. Specificity is achieved either by providing information on a smaller set than the overall population, of which the target case is a member, or when information can be coded, via causality, as information about the specific members of a given population. The base-rate fallacy is thus the result of pitting what seem to be merely coincidental, therefore low-relevance, base rates against more specific, or causal, information. A series of probabilistic inference problems is presented in which relevance was manipulated with the means described above, and the empirical results confirm the above account. In particular, base rates will be combined with other information when the two kinds of information are perceived as being equally relevant to the judged case.
• The base-rate fallacy and the difficulty of intrusion detection, Axelsson, S. (2000). The base-rate fallacy and the difficulty of intrusion detection.ACM Transactions on Information and System Security (TISSEC),3(3), 186-205. Many different demands can be made of intrusion detection systems. An important requirement is that an intrusion detection system beeffective; that is, it should detect a substantial percentage of intrusions into the supervised system, while still keeping the false alarm rate at an acceptable level. This article demonstrates that, for a reasonable set of assumptions, the false alarm rate is the limiting factor for the performance of an intrusion detection system. This is due to the base-rate fallacy phenomenon, that in order to achieve substantial values of the Bayesian detection rateP(Intrusion***Alarm), we have to achieve a (perhaps in some cases unattainably) low false alarm rate. A selection of reports of intrusion detection performance are reviewed, and the conclusion is reached that there are indications that at least some types of intrusion detection have far to go before they can attain such low false alarm rates.
• The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges, Koehler, J. J. (1996). The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges.Behavioral and brain sciences,19(1), 1-17. We have been oversold on the base rate fallacy in probabilistic judgment from an empirical, normative, and methodological standpoint. At the empirical level, a thorough examination of the base rate literature (including the famous lawyerengineer problem) does not support the conventional wisdom that people routinely ignore base rates. Quite the contrary, the literature shows that base rates are almost always used and that their degree of use depends on task structure and representation. Specifically, base rates play a relatively larger role in tasks where base rates are implicitly learned or can be represented in frequentist terms. Base rates are also used more when they are reliable and relatively more diagnostic than available individuating information. At the normative level, the base rate fallacy should be rejected because few tasks map unambiguously into the narrow framework that is held up as the standard of good decision making. Mechanical applications of Bayes's theorem to identify performance errors are inappropriate when (1) key assumptions of the model are either unchecked or grossly violated, and (2) no attempt is made to identify the decision maker's goals, values, and task assumptions. Methodologically, the current approach is criticized for its failure to consider how the ambiguous, unreliable, and unstable base rates of the real world are and should be used. Where decision makers' assumptions and goals vary, and where performance criteria are complex, the traditional Bayesian standard is insufficient. Even where predictive accuracy is the goal in commonly defined problems, there may be situations (e.g., informationally redundant environments) in which base rates can be ignored with impunity. A more ecologically valid research program is called for. This program should emphasize the development of prescriptive theory in rich, realistic decision environments.
• The base-rate fallacy and its implications for the difficulty of intrusion detection, Axelsson, S. (1999, November). The base-rate fallacy and its implications for the difficulty of intrusion detection. InProceedings of the 6th ACM Conference on Computer and Communications Security(pp. 1-7). ACM. Many different demands can be made of intrusion detection systems. An important requirement is that it beeffectivei.e. that it should detect a substantial percentage of intrusions into the supervised system, while still keeping thefalse alarmrate at an acceptable level.This paper aims to demonstrate that, for a reasonable set of assumptions, the false alarm rate is the limiting factor for the performance of an intrusion detection system. This is due to the base-rate fallacy phenomenon, that in order to achieve substantial values of the Bayesian detection rate,P(Intrusion|Alarm), we have to achievea perhaps unattainably lowfalse alarm rate.A selection of reports of intrusion detection performance are reviewed, and the conclusion is reached that there are indications that at least some types of intrusion detection have far to go before they can attain such low false alarm rates.
• Experience and the base-rate fallacy, Christensen-Szalanski, J. J., & Beach, L. R. (1982). Experience and the base-rate fallacy.Organizational Behavior and Human Performance,29(2), 270-278. This study shows that decision makers can use the base rate to assess posterior probabilities when they have experienced the relationship between the base rate and the diagnostic information. When they experience only the base rate, they do not use it.