Introduction to Bayes' Theorem
Bayes' theorem is a powerful result in probability that forms the basis for Bayesian inference. It has reshaped how probabilities and statistics are understood, and the simplest example will be illustrated in this discussion.
Conditional Probability Review
The probability of event A given event B is defined as
[ P(A|B)=\frac{P(A\text{ and }B)}{P(B)} . ]
Similarly, the probability of B given A is
[ P(B|A)=\frac{P(B\text{ and }A)}{P(A)} . ]
Because the intersection of A and B is the same as the intersection of B and A, the numerators in these two expressions are equal:
[ P(A\text{ and }B)=P(B\text{ and }A). ]
Derivation of Bayes' Theorem
Starting from the equality of the numerators, we can substitute the expression for (P(B|A)) into the formula for (P(A|B)):
[ P(A|B)=\frac{P(B|A)\,P(A)}{P(B)} . ]
This rearrangement shows how the two conditional probabilities are related through the ratio of the prior probabilities (P(A)) and (P(B)).
Bayes' Theorem Formula
The derived formula
[ P(A|B)=\frac{P(B|A)\times P(A)}{P(B)} ]
allows us to alternate between (P(A|B)) and (P(B|A)). In practice, we multiply the likelihood (P(B|A)) by the prior (P(A)) and then divide by the marginal probability (P(B)).
Utility of Bayes' Theorem
The theorem is especially helpful when one conditional probability is easy to evaluate while the other is difficult. By collecting real‑world data to estimate the easier probability and the priors, we can compute the harder probability without direct measurement.
Example: The Two‑Children Problem
Consider a couple with two children. Let
- (A) = “both children are girls” (2G)
- (B) = “at least one child is a girl” (1G)
We want (P(2G|1G)). Using Bayes' theorem:
[ P(2G|1G)=\frac{P(1G|2G)\times P(2G)}{P(1G)} . ]
- (P(1G|2G)=1) because if both are girls, there is certainly at least one girl.
- (P(2G)=\frac{1}{4}) (only the girl‑girl outcome out of four equally likely outcomes).
- (P(1G)=\frac{3}{4}) (girl‑girl, girl‑boy, boy‑girl all contain at least one girl).
Plugging the numbers in:
[ P(2G|1G)=\frac{1 \times \frac{1}{4}}{\frac{3}{4}}=\frac{1}{3}. ]
The result matches the calculation obtained directly from conditional probability, confirming the theorem’s correctness in this simple scenario.
Takeaways
- Bayes' theorem links two conditional probabilities and underpins Bayesian inference.
- By rearranging conditional probability definitions, the theorem is derived as P(A|B) = [P(B|A)·P(A)] / P(B).
- The theorem is especially useful when one conditional probability is easy to compute while the other is hard.
- In the classic two‑children problem, Bayes' theorem shows that the probability both children are girls given at least one girl is 1/3.
- Real‑world data can be collected to estimate the required probabilities, making Bayes' theorem applicable beyond textbook examples.
Frequently Asked Questions
Who is Dr. Trefor Bazett on YouTube?
Dr. Trefor Bazett is a YouTube channel that publishes videos on a range of topics. Browse more summaries from this channel below.
Does this page include the full transcript of the video?
Yes, the full transcript for this video is available on this page. Click 'Show transcript' in the sidebar to read it.
Helpful resources related to this video
If you want to practice or explore the concepts discussed in the video, these commonly used tools may help.
Links may be affiliate links. We only include resources that are genuinely relevant to the topic.