Overfitting in Deep Learning
The connection is indirect but important: augmentation increases dataset diversity, and that increased diversity is used to help prevent overfitting.
Why One Dataset Is Not Enough
When a computer vision model is trained with a small dataset, the original collection may not provide enough variety. Data augmentation addresses this by using existing images as starting points and presenting them in varied forms. The goal is to create a larger and more diverse set of training examples, helping prevent overfitting.
From One Image to Many Variations
Think of the original image as the input to a preparation process. An augmentation configuration determines which transformations are available, such as rotation, shifting, or flipping. Those transformations produce varied training examples from the original image. Across a small collection of images, this process increases both the apparent size and the diversity of the training dataset.
Following One Image Through Augmentation
Suppose a training image is prepared with an augmentation configuration that makes rotation, shifting, and flipping available. What changes in the training data?
- 1Start with the existing image
The original image is the starting point rather than a newly collected image.
- 2Apply configured transformations
The configuration determines which transformations are available while the training data is prepared.
- 3Present varied examples
The model receives varied forms of the existing image, increasing the diversity of the training examples.
The original collection can supply a larger and more diverse set of training examples for model training.
Why Variation Helps Generalization
The purpose of augmentation is to increase the diversity of the training data and help prevent overfitting. Training with varied forms of existing images gives the model a broader set of training examples than the original collection alone. This encourages the training process to use the available variety instead of depending only on the limited forms in the small dataset.
The connection is indirect but important: augmentation increases dataset diversity, and that increased diversity is used to help prevent overfitting.
Reading the Generator Configuration
ImageDataGenerator is configured by passing named augmentation parameters into its call. The resulting configured generator is stored under the name datagen in the source configuration. To read this kind of statement, move from left to right: identify the object being called, inspect each named argument and its assigned value, and then identify the name receiving the result.
| Configuration element | Meaning in the configuration |
|---|---|
| ImageDataGenerator | The object being called to configure augmentation |
| rotation_range | A named augmentation parameter supplied to ImageDataGenerator |
| horizontal_flip | A named augmentation parameter supplied to ImageDataGenerator |
| Assigned value | The value written after a parameter name |
| datagen | The name receiving the configured generator |
Elements explicitly identified in the supplied source description
Tracing Data Through ImageDataGenerator
The configured ImageDataGenerator represents the selected augmentation configuration. Original images are supplied as the starting material, the configured transformations are applied while the training data is prepared, and the resulting varied examples are made available for model training. The important distinction is between the generator configuration and the later training process: a problem in the named arguments is different from a problem elsewhere in training.
When debugging, trace the configuration from left to right. First identify the object being called. Next inspect every named argument and its assigned value. Finally check the name receiving the result. This helps separate a configuration problem from a problem elsewhere in the training process.
Limits of the Supplied Configuration
Common mistake Treating a parameter name as if it were its value
What it looks likeReading rotation_range as the selected setting rather than as the name of a setting
Why it failsThe source emphasizes separating each named parameter from the value assigned to it.
Do this insteadIdentify the name first, then inspect the value associated with that name.
Common mistake Assuming augmentation creates an unrelated image collection
What it looks likeDescribing the augmented dataset as if it came only from newly collected images
Why it failsAugmentation starts with existing images and applies transformations to them.
Do this insteadTrace each varied example back to an existing training image and the configured transformations.
Common mistake Ignoring the name receiving the configured generator
What it looks likeInspecting only the arguments passed into ImageDataGenerator
Why it failsThe source configuration stores the configured generator as datagen, and debugging requires checking where the result is assigned.
Do this insteadAfter reviewing the named arguments, check the receiving name as the final step of the left-to-right trace.
Common mistake Claiming that augmentation guarantees prevention of overfitting
What it looks likeSaying that using augmentation always eliminates overfitting
Why it failsThe stated purpose is to help prevent overfitting, not to guarantee that it cannot occur.
Do this insteadDescribe augmentation as a way to increase diversity and help prevent overfitting.
Check Your Understanding
A configuration passes named arguments rotation_range and horizontal_flip into ImageDataGenerator, and the resulting object is assigned to datagen. Explain which items are parameter names, which items are assigned values, and which name receives the configured generator. Then explain why this configuration can help with a small image dataset.
Hints
- A parameter name identifies a configurable aspect of augmentation.
- An assigned value appears beside its parameter name.
- The receiving name is the name assigned the result of the ImageDataGenerator call.
- Connect the purpose to increased diversity and help with preventing overfitting.
Practice Solution
Classify the elements in the described configuration.
- 1Identify the called object
ImageDataGenerator is the object being called to receive augmentation parameters.
- 2Identify parameter names
rotation_rangeandhorizontal_flipare the named parameters explicitly identified in the configuration. - 3Identify assigned values
Each parameter has a value assigned to it in the configuration. The value is distinct from the parameter name.
- 4Identify the receiving name
datagen is the name used to store the configured generator.
- 5Explain the purpose
The selected transformations can create varied training examples from existing images, increasing dataset diversity and helping prevent overfitting.
Read the configuration as object being called, named arguments with assigned values, and the name receiving the configured result.
Key Takeaways
- 1Data augmentation creates varied training examples from existing images.
- 2Applying transformations can artificially increase the size and diversity of a small training dataset.
- 3ImageDataGenerator accepts named augmentation parameters, including
rotation_rangeandhorizontal_flip. - 4Read a configuration by separating each parameter name from its assigned value and then checking the name receiving the result.
- 5The purpose of augmentation is to increase training-data diversity and help prevent overfitting.
At a glance
- Data augmentation uses transformations of existing images to create varied training examples.
- The resulting training data is larger and more diverse than the original collection alone.
- ImageDataGenerator is configured with named parameters such as
rotation_rangeandhorizontal_flip. - A configuration should be read from left to right: called object, named arguments and values, then receiving name.
- Increased data diversity is intended to help prevent overfitting.