Bayes' Theorem

Advanced

Bayes' theorem is a mathematical rule for updating the probability of an event or belief when new evidence becomes available, by combining the original probability with how likely the evidence would be if the belief were true.

Bayes' theorem, named after eighteenth-century mathematician Thomas Bayes, shows how to reverse conditional probabilities. It is especially useful when evidence is imperfect.

For example, suppose 2% of transactions are fraudulent. A fraud detection system correctly flags 90% of fraudulent transactions but also wrongly flags 5% of genuine ones. If a transaction is flagged, the probability it is actually fraudulent is not 90%. Out of every 1,000 transactions, about 18 frauds are flagged and about 49 genuine transactions are wrongly flagged. So only about 27% of flagged transactions are fraudulent.

This result, which surprises many people, shows how important the starting probability, called the prior, is. When the event is rare, even accurate tests produce many false alarms.

Bayes' theorem is used in fraud detection, credit scoring, medical testing, spam filters, machine learning and investment analysis. Bayesian thinking more broadly means updating beliefs gradually as new information arrives, rather than ignoring evidence or overreacting to it.

For worked examples and full lessons, visit notesbysara.org.