📌 Daily Briefing

Linear regression & the mean squared error loss — learning as line-fitting

Intuition. Linear regression is the simplest complete machine-learning system: it has a model, a loss, and a way to learn. The model says a prediction is just a weighted sum of inputs: ŷ = w·x + b. Predicting apartment rent from square meters? Find the slope w and intercept b that make the line pass as close as possible to the data points. Every neural network is, at its core, many of these linear units stacked with nonlinearities in between.

The mechanism. "As close as possible" needs a number — the loss. Mean squared error averages the squared gaps between predictions and truth: MSE = (1/n) Σ (ŷᵢ − yᵢ)². Squaring makes errors positive, punishes large misses disproportionately, and — crucially — makes the loss a smooth bowl-shaped (convex) function of w and b, so it has exactly one minimum. This connects directly to yesterday's MLE: minimizing MSE is exactly maximum-likelihood estimation under the assumption of Gaussian noise.

Finding the minimum. Because the bowl is smooth, gradients (phase 1!) tell us which way is downhill: ∂MSE/∂w = (2/n) Σ (ŷᵢ − yᵢ)·xᵢ. Take small steps against the gradient and you slide to the bottom — that's gradient descent, tomorrow's natural follow-up. (For plain linear regression there's even a closed-form solution, but the gradient view is the one that scales to GPT.)

import numpy as np
X = np.array([30, 50, 70, 90.])      # m²
y = np.array([700, 1000, 1350, 1600.]) # rent €
w, b, lr = 0.0, 0.0, 0.0003
for _ in range(200000):
    err = (w*X + b) - y              # prediction gap
    w -= lr * 2 * np.mean(err * X)   # gradient step
    b -= lr * 2 * np.mean(err)
print(f"rent ≈ {w:.1f}·m² + {b:.1f}")  # ≈ 15.2·m² + 250

Why it matters. The pattern model → loss → gradient → update is the entire template of deep learning. Change the model from a line to a transformer and the loss from MSE to cross-entropy, and the same loop trains an LLM. Master this small machine and you've seen the skeleton of every large one.

Curriculum: AI Engineering from Scratch