Welcome back
Last time, we unrolled a map of meaning. Embeddings give each word a position, so similar words sit close together, and the network learns that map by itself.
Today we turn from words to pictures. How might a network look at a photograph and spot a bird?
What a photo looks like to a network
To a computer, a photo is just a grid of numbers. Each tiny square, or pixel, holds a few values for its brightness and colour. A modest photo might be 1,000 pixels wide and 1,000 tall, which makes a million pixels, each with three colour values.
We could feed all those numbers into an ordinary layer, where every neuron listens to every pixel. But that would mean billions of weights for one layer. It would also be strangely forgetful. A bird in the top-left corner and the same bird in the bottom-right would look completely different to it, because entirely different weights would see each one.
There’s a better way, and any birdwatcher already knows it.
The patient watcher
Picture yourself at the edge of a meadow at dawn, binoculars in hand, facing a long hedgerow. You don’t take in the whole hedge at once. You move your gaze slowly, one small patch at a time, left to right, row by row.
And in every patch you’re looking for the same things: a flash of red, a curved beak, a striped wing. You don’t use one set of clues for the left end of the hedge and another for the right. A red wing is a red wing wherever it appears.
The convolutional network
A convolutional network (often called a CNN) looks at images just as the watcher looks at the hedge.
The heart of it is a filter: a small square of weights, perhaps 3 pixels by 3. The filter is the watcher’s trained eye for one particular clue. It slides across the whole image, patch by patch. At each spot, it multiplies its weights by the pixels underneath and adds them up. This is our familiar weighted sum from Lesson 3, applied to one small patch. The result tells us how strongly that clue appears right there.
As the filter slides over every position, it builds a new grid called a feature map. Picture a heat map of where the clue was found. If the filter has learned to spot vertical edges, its feature map glows wherever vertical edges appear.
Two ideas make this powerful:
- Looking locally. Each step considers only a small patch, because nearby pixels belong together.
- Sharing weights. The same filter is reused across the whole image. A clue learned in one corner is recognised everywhere, and the network needs only a handful of weights for each filter, instead of millions.
Many eyes at once
One filter spots one kind of clue. So a convolutional layer uses many filters side by side, perhaps 32 or 64. One may learn to notice edges, another a patch of colour, another a soft gradient. Nobody tells them what to look for. As always, their weights start random and are shaped by the loss, gradient descent and backpropagation.
What to carry with you
A convolutional network slides small filters across an image, patch by patch, reusing the same weights everywhere. Each filter learns to spot one kind of local pattern, wherever it appears.