• Distinguish prior probability from posterior probability
• Carry out simple Bayes' theorem calculations
Deeper study
• Fully understand the Bayes' theorem formula
• Use the law of total probability
• Apply Bayes' theorem to real-life problems
• Avoid the common mistakes
Real-world practice
• Understand the naive Bayes classifier
• Learn the basics of Bayesian networks
• Perform Bayesian updating
• Apply Bayes' theorem in AI and data science
At a glance
Bayes' theorem is the conditional-probability rule P(A|B) = P(B|A)P(A)/P(B), which tells you how to update a prior probability P(A) into a posterior probability P(A|B) once new evidence B has been observed.
For a disease with 0.1% prevalence and a test with 95% sensitivity and 95% specificity, a positive result means only about a 1.87% chance of actually having the disease (the prior, or base rate, matters enormously)
The evidence probability in the denominator is computed with the law of total probability: P(B) = P(B|A)P(A) + P(B|¬A)P(¬A)
Bayes' theorem is a core tool across AI and data science, from naive Bayes spam filters to medical diagnosis and recommender systems
Bayes' theorem is an application of conditional probability, so that concept is essential
Basic Concepts of Probability
You need the basic properties and rules of probability to understand Bayes' theorem
What Is Bayes' Theorem?
Difficulty undefined/5
About 30 min
Bayes' theorem is a mathematical formula that tells you how to update an existing belief (the prior probability) when new information (evidence) arrives.
🔍 Think of a detective. Every time a new clue turns up at the crime scene, the probability that a given suspect is the culprit changes. At the start everyone is equally suspicious (the prior probability), but with each piece of evidence, a fingerprint, a DNA match, an alibi, the probability gets updated (the posterior probability). Bayes' theorem is simply the mathematical expression of this updating process.
Key points
•
A formula for updating a prior probability into a posterior probability
•
A method for revising your beliefs in light of new evidence
•
Loading...
•
Used everywhere in real life: medical diagnosis, AI, spam filters, and more
•
Discovered by Thomas Bayes (1701–1761), an 18th-century English Presbyterian minister and statistician, and submitted to the Royal Society posthumously by Richard Price in 1763. Pierre-Simon Laplace generalized it in his 1812 "Théorie Analytique des Probabilités"
A simple example
Medical test example: intuition vs. the Bayesian calculation
🔢 Assumptions
• Disease prevalence (prior probability): 0.1% (1 in 1,000 people)
• Test sensitivity: 95% (the chance a sick person tests positive)
• Test specificity: 95% (the chance a healthy person tests negative → a 5% false-positive rate)
❓ If you test positive, what is the probability that you actually have the disease?
❌ Intuition: "The test is 95% accurate, so I'm 95% likely to have it, right?"
✅ Reality: about 1.87% (roughly 2%)
📊 Thought experiment: test 1,000 people
• 1 person actually has the disease × 95% ≈ 0.95 true positives
• 999 healthy people × 5% ≈ 49.95 false positives
• Total positives ≈ 50.9, of which only ≈ 0.95 are truly sick
• Probability of disease given a positive result = 0.95 ÷ 50.9 ≈ 1.87%
💡 When the prevalence (prior probability) is very low, even an accurate test leaves a low probability of disease after a positive result. Bayes' theorem is the tool that exposes exactly this gap between intuition and reality.
Check your understanding
Answer:
It is the initial probability before you obtain any new information or evidence.
Example: the general probability of having the disease before taking the test (the prevalence).
Answer:
No. The actual probability depends heavily on the prior probability (the prevalence).
If the disease is rare, the probability of infection can still be low even after a positive result.
Answer:
Because the prevalence (prior probability) is extremely low at 0.1%.
Among 1,000 people only 1 is sick and 999 are healthy, so the 5% false positives from the healthy group (about 50 people) outnumber the 95% true positives from the sick group (about 1 person) by roughly 50 to 1.
As a result, of the roughly 51 positive results, only 1 is a true patient → about 2%.
💡 The key insight of Bayes' theorem is that when the prior is small, the denominator (the pool of false positives) becomes far larger than the numerator (the true positives).
Answer:
① P(A), the prior: the probability you assigned before seeing the evidence
② P(B|A), the likelihood: the probability of observing evidence B if A is true
③ P(B), the evidence: the overall probability of observing B (computed with the law of total probability)
④ P(A|B), the posterior: the updated probability of A after seeing evidence B
Let's look at the mathematical structure of Bayes' theorem and how to actually compute with it.
Bayes' theorem:
Loading...
What each term means:
- Loading...: the prior, the probability that A is true before any evidence
- Loading...: the likelihood, the probability of observing B given that A is true
- Loading...: the evidence (normalizing constant), the overall probability of observing B
- Loading...: the posterior, the probability that A is true after observing B
Computing P(B) with the law of total probability:
Loading...
Mathematical formula
Posterior = (Likelihood × Prior) ÷ Evidence
Learn through examples
Example 1: Interpreting a COVID-19 Test Result
The probability of actually being infected after a positive PCR test
Given:
• Local infection rate: 1% → Loading...
• Test sensitivity: 95% → Loading...
• Test specificity: 95% → Loading...
We want: Loading...
Answer:
Step 1: Compute Loading...
Loading...
Loading...
Step 2: Apply Bayes' theorem
Loading...
Even with a 95% accurate test, the probability of actually being infected after a positive result is only 16%. The reason is the low infection rate.
Example 2: Spam Filtering
The probability that an email containing the word "free" is spam
Data:
• Share of all email that is spam: 40% → Loading...
The single word "free" is enough to classify an email as spam with 84% probability.
Example 3: DNA Evidence in a Criminal Investigation
The probability that a suspect is the culprit given a DNA match
Scenario:
• Share of the city's population who could be the culprit: 0.01% → Loading...
• Probability the culprit's DNA matches: 100% → Loading...
• Probability an innocent person's DNA matches by chance: 0.1% → Loading...
Answer:
Loading...
Loading...
Even with a DNA match, the probability that the suspect is the culprit is only 9%. Additional evidence is needed.
Common mistakes
Mistake:
Confusing Loading... with Loading...
Why is it wrong?
"The probability of testing positive if you have the disease" and "the probability of having the disease if you test positive" are completely different quantities
Correct approach:
Always be clear about which event is the condition and which is the outcome. Bayes' theorem is precisely the bridge between the two.
Mistake:
Ignoring the prior and looking only at the likelihood
Why is it wrong?
Judging the result from the test's accuracy (the likelihood) alone
Correct approach:
When the prior (prevalence) is low, even a highly accurate test can leave a low probability of disease after a positive result.
Mistake:
Skipping the calculation of the denominator P(B)
Why is it wrong?
Estimating by intuition instead of using the law of total probability
Correct approach:
Always compute P(B) exactly with the law of total probability.
Practice questions
Hint:
Use P(A|defective) = P(defective|A) × P(A) / P(defective).
IBM Watson Health's cancer diagnosis support system
Why it matters:
It combines symptoms, test results, and patient history to compute disease probabilities. Bayes' theorem integrates each piece of evidence systematically to support the physician's diagnosis.
Technology
Spam filtering and email classification
Example:
Gmail's spam filter
Why it matters:
Trained on billions of emails, it computes the probability of spam from words, sender, subject line, and more. A naive Bayes classifier is the core algorithm.
Finance
Credit risk assessment
Example:
Automated loan approval systems
Why it matters:
They compute the probability of default from a customer's income, credit score, borrowing history, and other data, updating the risk estimate whenever new information arrives.
Marketing
Predicting customer behavior
Example:
Amazon's recommendation system
Why it matters:
It predicts the probability of the next purchase from a customer's past purchases, searches, and clicks, collecting behavioral data in real time to improve recommendations.
Case study: The Bayesian Approach in Netflix's Recommendation Algorithm
Background
Netflix needs to recommend personalized content to more than 200 million users.
The problem
How can recommendations be accurate when there is little data about a new user or a new title?
The data
• Users' viewing history
• Rating data
• Completion rates
• Genre, cast, and director information
• Preferences of similar users
Method
Netflix takes a Bayesian approach: it starts from a prior (average preference by genre) and keeps updating it with each user's behavioral data.
Solution and results
1. Set the prior: the average genre preferences of a typical user
2. Compute the likelihood: the characteristics of the content the user has watched
3. Update the posterior: refresh the preference estimate after every viewing
4. Generate recommendations: suggest the content with the highest posterior probability
Netflix has reported that more than 75% of viewing comes through its recommendation algorithm. The Bayesian approach makes relevant recommendations possible for new users from day one.
Conclusion:
Bayes' theorem is the ideal framework for learning incrementally under uncertainty. The more data accumulates, the more accurate it becomes.
References
Bayes, T. & Price, R. (1763), "An Essay towards Solving a Problem in the Doctrine of Chances", Philosophical Transactions of the Royal Society of London 53, 370–418 (DOI 10.1098/rstl.1763.0053) — the original paper on Bayes' theorem, submitted to the Royal Society by Richard Price after Bayes' death
Laplace, P.-S. (1812), "Théorie analytique des probabilités", Courcier, Paris — the classic work that generalized Bayes' theorem and laid the mathematical foundations of probability theory (cited in the key points above)
Gigerenzer, G. & Hoffrage, U. (1995), "How to Improve Bayesian Reasoning Without Instruction: Frequency Formats", Psychological Review 102(4), 684–704 — shows how natural frequencies (like the 1,000-person thought experiment in the medical test example) correct base-rate neglect
Sahami, M., Dumais, S., Heckerman, D. & Horvitz, E. (1998), "A Bayesian Approach to Filtering Junk E-Mail", AAAI Workshop on Learning for Text Categorization, Technical Report WS-98-05, 55–62 — the landmark original paper on the naive Bayes spam filter discussed in this page
