📌 Daily Briefing

Bayes' Theorem — updating beliefs with evidence

Intuition. Bayes' theorem is the mathematically correct way to change your mind. You start with a prior belief about how likely something is, observe evidence, and compute an updated (posterior) belief. It answers the question we actually care about — "given what I just saw, how likely is my hypothesis?" — from the quantity that's usually easier to measure: "if my hypothesis were true, how likely would this observation be?"

The math. For a hypothesis H and data D:

P(H | D) = P(D | H) · P(H) / P(D)

posterior = likelihood × prior / evidence

P(D) is just a normalizer: sum P(D|H)·P(H) over all competing hypotheses. The classic trap is ignoring the prior. Example: a disease affects 1% of people; a test is 95% accurate. A positive result does not mean 95% chance of disease — it means 0.95×0.01 / (0.95×0.01 + 0.05×0.99) ≈ 16%, because true positives from a rare disease are swamped by false positives from the healthy majority.

# Bayes in five lines
prior = 0.01                      # P(disease)
sens, fpr = 0.95, 0.05            # P(+|disease), P(+|healthy)
evidence = sens*prior + fpr*(1-prior)   # P(+)
posterior = sens*prior / evidence
print(round(posterior, 3))        # 0.161

Why it matters for AI engineering. Bayes is everywhere once you look: Naive Bayes classifiers score spam by multiplying per-word likelihoods; Bayesian optimization tunes hyperparameters by maintaining a posterior over the loss surface; and the whole "probabilistic view" of ML treats training as computing a posterior over model parameters given data. Even LLM behavior is often analyzed through a Bayesian lens — in-context learning can be viewed as the model implicitly updating on the examples in your prompt. Whenever you evaluate a model and ask "the eval passed, so how likely is the system actually good?", you're doing a Bayes update — and base rates matter just as much as they do for medical tests.

Curriculum: AI Engineering from Scratch · Phase 2: Probability & Statistics