Image Classification with Deep Learning

Feature extraction separates two jobs: the convolutional base transforms images into learned visual features, while a new classifier interprets those features for the new classes.

Programming

Starting with Learned Vision

Training an image classifier from the beginning requires the model to learn visual representations as well as the final class decisions. A pretrained network gives you a different starting point: it is a saved network that was previously trained on a large image dataset, such as ImageNet. Instead of discarding what that network learned, you can reuse the part that produces visual representations and train a new classifier for your own image classes.

Key point

Feature extraction separates two jobs: the convolutional base transforms images into learned visual features, while a new classifier interprets those features for the new classes.

How can representations learned from a large dataset help classify images in a new task?

Following an Image Through the Base

Feature extraction begins before the new classifier is trained. First, a new image is passed through the retained convolutional and pooling layers of the pretrained network. Those layers progressively transform the image into a learned visual representation. The output of the convolutional base then becomes the input to a separate classifier trained for the new image problem.

How does a new image move through successive convolutional layers, and what feature representation comes out at the end?

Worked example

A New Two-Class Image Problem

Suppose a new task must distinguish between two image classes using a pretrained VGG16 network.

1
Select the base
Keep the VGG16 convolutional and pooling layers rather than its original densely connected classification section. In this feature-extraction setup, the selection is represented by include_top=False.
2
Process the new image
Pass the new image through the retained convolutional base. The base transforms the image into a learned visual representation.
3
Train for the new task
Use the base output as input to a new classifier trained for the two new classes.

Result

The original visual feature-producing part is reused, while the classifier responsible for the original classes is replaced by a task-specific classifier.

Separating Base and Classifier

The boundary between reusable and task-specific parts is placed between the convolutional base and the original classifier. The convolutional base contains convolution and pooling layers that produce learned visual representations. The original classifier is tied to the classes from the first training task, so it should generally be replaced with a new classifier for the new class set.

Which parts of a pretrained network are reused, which part is replaced or retrained, and how are they connected?

Choosing a Layer Depth

Layer depth affects how general or task-specific a learned feature tends to be. Earlier convolutional layers usually capture local visual patterns such as edges, colors, and textures. Deeper layers capture increasingly abstract concepts, such as a cat ear or a dog eye. Because early features are more generic, they can be more reusable when the new dataset differs substantially from the original training data.

What changes from early to late convolutional layers, and why are early features generally more reusable across tasks?

Part of the network	Typical representation	Reuse implication
Earlier convolutional layers	Edges, colors, and textures	More generic and often more reusable
Deeper convolutional layers	More abstract concepts, such as a cat ear or dog eye	May be less reusable when the new task differs substantially
Original classifier	Representations tied to the original classes	Generally replaced for the new task

Depth changes the kind of visual information represented and influences the reuse boundary.

Avoiding Reuse Errors

Common mistake
Keeping the original classifier unchanged for a new set of image classes.
What it looks like
Treating the original classifier as the complete solution after reusing the convolutional base.
Why it fails
The original classifier is tied to the classes from the first training task, not necessarily the classes in the new problem.
Do this instead
Replace the original classifier with a new classifier trained using the output of the retained convolutional base.
Common mistake
Including the original densely connected classification section when the goal is feature extraction.
What it looks like
Selecting the full VGG16 network instead of selecting the convolutional base without the original top section.
Why it fails
The feature-extraction setup requires the new images to pass through the retained convolutional and pooling layers before a new classifier uses the resulting output.
Do this instead
Use the feature-extraction boundary represented by include_top=False.
Common mistake
Assuming the deepest available features are always the best features to reuse.
What it looks like
Using higher-layer representations without considering how different the new dataset is from the original dataset.
Why it fails
Deeper layers encode increasingly abstract concepts and may be less reusable for a substantially different task.
Do this instead
Consider earlier convolutional layers when the new dataset differs substantially, because their features are more general.
Common mistake
Describing feature extraction as if the new classifier processes the raw image directly.
What it looks like
Skipping the convolutional base in the explanation of the data path.
Why it fails
The convolutional base transforms the image into learned visual features before the new classifier interprets them.
Do this instead
Trace the image through the retained convolutional and pooling layers, then connect the base output to the new classifier.

What can go wrong if the wrong layers are selected or the original classifier is reused unchanged?

Applying the Decision

What do you think happens?

A new image task uses classes that differ substantially from the classes in the dataset used to train the pretrained network. Which part is generally the safer starting point for reuse: an earlier convolutional layer or a deeper convolutional layer?

An earlier convolutional layer
A deeper convolutional layer
The original classifier

Reveal answer

Answer:

An earlier convolutional layer

Earlier layers tend to capture more generic patterns such as edges, colors, and textures. Deeper layers represent more abstract concepts and may be less reusable when the new dataset differs substantially.

EASY

Explain the data path for a new image classification task in three parts: identify the reused component, describe the representation it produces, and state what component is trained for the new classes.

Hints

Start with the convolutional base rather than the original classifier.
Mention the convolutional and pooling layers.
End with the new classifier that interprets the base output.

Your attempt (not saved)

MEDIUM

A new dataset is very different from the original training dataset. Decide whether earlier or deeper convolutional features are more appropriate as a starting point, and justify your choice using the kinds of patterns those layers represent.

Hints

Compare generic local patterns with abstract concepts.
Consider how similarity between the two datasets affects reuse.

Your attempt (not saved)

Essential Takeaways

1
A pretrained network is a saved network previously trained on a large dataset, such as ImageNet.
2
Feature extraction sends new images through a retained convolutional base and uses the resulting visual representation as input to a new classifier.
3
The convolutional base contains reusable convolutional and pooling layers, while the original classifier is tied to the original classes and is generally replaced.
4
Earlier layers usually learn generic patterns such as edges, colors, and textures; deeper layers learn more abstract concepts.
5
When the new dataset differs substantially from the original one, earlier convolutional features may provide a more appropriate starting point for reuse.

At a glance

A pretrained network provides learned visual representations from earlier training on a large image dataset.
Feature extraction separates visual representation from task-specific classification.
The convolutional base is reused, while a new classifier is trained for the new classes.
Feature reuse depends on layer depth and on how similar the new dataset is to the original dataset.
Earlier convolutional features are generally more generic, whereas deeper features are more abstract and task-related.

More concepts starting with I