AI Models lessons

Learning from examples

In normal programming, a person writes the rules. In machine learning, the computer works out the rules from examples.

Two flow chains light up in turn: rules and data give answers; data and answers give a model.

When every example comes with its right answer, called a label, it is supervised learning. With no labels, the model hunts for patterns on its own, which is unsupervised learning.

Practice: Labelled or unlabelled?

Sort tasks into supervised and unsupervised.

Inside a neuron

A neuron takes in numbers called inputs. It multiplies each input by a weight, adds them up, then adds a bias.

Weighted sum: z = \class{ix-a}{w_{1}}\class{ix-c}{x_{1}} + \class{ix-a}{w_{2}}\class{ix-c}{x_{2}} + \class{ix-b}{b}

A big weight means that input matters a lot. A negative weight pulls the total down, and the bias shifts it whatever the inputs are.

A neuron with two inputs; sliders change the weights and bias, line thickness and the sum update live.

Learning means finding good values for every weight and bias. A large model has billions of them.

Practice: Compute the weighted sum

Type a neuron's weighted sum from random inputs, weights and bias.

Activation functions

Next, the neuron passes z through an activation function. This bends the straight-line sum into a curve.

Without that bend, stacking layers would still only make straight lines. With it, a network can learn curved, complicated patterns.

1. \class{ix-a}{\text{ReLU}}(z) = \max(0, z) (keeps positives, turns negatives into 0)
2. \class{ix-b}{\sigma}(z) = \frac{1}{1 + e^{-z}} (squashes any z to between 0 and 1)
3. \class{ix-d}{\tanh}(z) = \frac{e^z - e^{-z}}{e^z + e^{-z}} (squashes any z to between -1 and 1)

Plot of ReLU, sigmoid and tanh in turn, with z on a slider and the output read off.

ReLU and close cousins of it are the usual choice inside a network. Sigmoid suits an output that should be a probability.

Practice: Match the activation

Match activation names to their graphs, then output ranges to their formulas.

Layers and the forward pass

Neurons sit in layers. The input layer holds the data, the hidden layers transform it and the output layer gives the prediction.

Each neuron's output becomes an input to the next layer. Working through from start to end is called the forward pass.

A 2-2-1 network computed one neuron at a time; one hidden neuron is switched off by ReLU.

The prediction is written \hat{y} and read as "y hat". The right answer is plain \class{ix-d}{y}.

Practice: Trace the signal

Type the output of a small network with one ReLU neuron; random weights.

Measuring mistakes with loss

To learn, a model must know how wrong it is. A loss function turns all its mistakes into one number.

For number answers, the usual choice is the mean squared error. Square each error, then take the average.

Mean squared error: \text{MSE} = \frac{1}{n}\sum_{i=1}^{n}(\class{ix-d}{y_i} - \hat{y}_i)^2

1. \frac{1}{2}\left((\class{ix-d}{4} - 3)^2 + (\class{ix-d}{2} - 5)^2\right) (answers 4 and 2, predictions 3 and 5)
2. \frac{1}{2}(1 + 9) (square each error)
3. \text{MSE} = 5 (the average)

Data points and a prediction line; each error is drawn as a square and the MSE updates as the line changes.

Squaring makes every error positive. It also punishes big misses far more than small ones.

Practice: Fit the line

Move weight and bias sliders until the loss of a line through random points drops below a target.

Gradient descent

Gradient descent is how a model lowers its loss. For each weight, it finds the slope of the loss and steps downhill.

Update rule: \class{ix-a}{w_{\text{new}}} =\class{ix-a}{w} - \alpha \frac{\partial L}{\partial \class{ix-a}{w}}

The slope \frac{\partial L}{\partial w} is the gradient, the same idea as a derivative in Calculus. The learning rate \alpha sets how big each step is.

A weight rolls down a loss curve in steps, with the learning rate on a slider and the loss after each step plotted below.

Too small a learning rate and training crawls. Too big and the weight leaps over the bottom, so the loss climbs.

Practice: Update a weight

Apply the update rule to a random weight, gradient and learning rate.

Practice: Read the loss curve

Three loss curves on one plot; questions on which learning rate each one shows.

Backpropagation

A big network has billions of weights, and each one needs its gradient. Backpropagation finds them all in one sweep backwards from the loss.

Each step back multiplies by one local slope. This is the chain rule from Calculus.

1. \frac{\partial L}{\partial w} = \frac{\partial L}{\partial a} \cdot \frac{\partial a}{\partial z} \cdot \frac{\partial z}{\partial w} (chain rule)
2. = 2(a - \class{ix-d}{y}) \cdot 1 \cdot \class{ix-c}{x} (one local slope each; ReLU has slope 1 when z > 0)
3. = 2(3 - \class{ix-d}{4}) \cdot 1 \cdot \class{ix-c}{2} = -4 (with x = 2, y = 4 and a = 3)

A forward pass fills in each value, then the backward pass fills in each gradient, then the weight updates.

A negative gradient means raising \class{ix-a}{w} lowers the loss. Gradient descent then takes that step.

The training loop

Training repeats a few steps millions of times. Each pass uses a small group of examples called a batch.

One full pass through all the training data is called an epoch.

Order the steps of the training loop, then the steps inside one neuron.

Convolutional networks

A picture is a grid of numbers, one per pixel. A convolutional neural network (CNN) slides a small grid of weights, called a filter, across it.

At each spot it multiplies the overlapping numbers and adds them up. A big total means the filter's pattern is there.

A 3 by 3 edge filter slides over a 6 by 6 image and fills in the feature map one cell at a time.

Early filters pick out edges. Deeper layers combine edges into shapes such as eyes, wheels and letters.

Practice: Apply the filter

Multiply a 3 by 3 patch by a filter and type the sum; random patches and filters.

Recurrent networks

Text, speech and prices arrive in order. A recurrent neural network (RNN) reads them one step at a time and keeps a hidden state as its memory.

One RNN step: h_t = \tanh(\class{ix-a}{w_{h}}h_{t-1} + \class{ix-a}{w_{x}}\class{ix-c}{x_t} + \class{ix-b}{b})

Each step mixes the new input with the old memory. Over a long passage, the early words fade.

An RNN reads a sentence word by word; its memory shows the earlier words fading.

Attention

Attention lets every word look straight at every other word, however far away. It then decides how much each one matters.

Each word has a query (what it is looking for), a key (what it offers) and a value (what it passes on). Comparing queries with keys gives a score for every pair.

Scaled dot-product attention: \text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V

Softmax turns the scores into weights that add up to 1. Each word's output is a mix of the values, using those weights.

Attention weights from "it" to every other word, for two sentences that change what "it" means.

Practice: Memory and attention

Questions on RNN memory, softmax weights and which word attention should pick.

Transformers

A transformer stacks attention layers with small feed-forward networks, dozens of times over. It reads every word at once, so it trains fast on modern chips.

Large language models such as GPT, Claude and Gemini are transformers. They write by predicting the next token (a word or part of a word), adding it, and repeating.

A prompt passes through a transformer's stages, ending in next-token probabilities.

Training vs inference

Training is when the model learns and its weights change. For a large model it runs for weeks or months on tens of thousands of chips.

Inference is when the finished model is used. The weights are frozen and only the forward pass runs, so each answer takes seconds.

Training and inference shown in turn: the loop that changes weights, and the forward pass that only uses them.

Every message you send to a chat assistant is inference. The learning happened long before you typed.

Practice: Training or inference?

Sort statements into training and inference.