Training and Validation Loss
A callback turns training from a process that only runs toward a fixed endpoint into a process that can observe training behavior and respond while the model is fitting.
Try it: Training a Keras-Style Model: History and Early Stopping
The Keras workflow on a tiny dense network: stack Dense layers (the summary counts units * (inputs + 1) parameters per layer), compile with an optimizer, loss and accuracy metric, fit for a number of epochs in mini-batches with validation data, and read the history. Training loss keeps falling while validation loss reaches its lowest point and then rises: the model starts to overfit, and EarlyStopping with patience can stop there and restore the best weights.
How it works
- Build a Sequential stack of Dense layers ending in Dense(1, sigmoid); each layer has units * (inputs + 1) weights and biases, initialised Glorot-uniform with zero biases.
- compile(): mini-batch SGD with the chosen learning rate, binary cross-entropy (or mean squared error) loss, and accuracy (prediction > 0.5) as the metric.
- fit(): each epoch walks the training set in batches of batch_size; every batch runs a forward pass, backpropagates the mean batch loss and updates every weight .
- After each epoch the history records the epoch's training loss and accuracy (batch-weighted average of values seen during the epoch) and the val_loss and val_accuracy on the held-out validation points.
- EarlyStopping(monitor="val_loss", patience) keeps the best val_loss; after patience epochs with no improvement it stops training, and restore_best_weights puts back the weights of the best epoch.
- evaluate() reports loss and accuracy on the validation data with the final weights.
Default run (153 steps): Sequential([Dense(16, "relu"), Dense(16, "relu"), Dense(1, "sigmoid")]) on inputs of shape (2,). model.summary(): dense 16 * (2 + 1) = 48; dense_1 16 * (16 + 1) = 272; dense_2 1 * (16 + 1) = 17; total 337 trainable parameters, initialised with Glorot uniform (biases 0). … model.evaluate(x_val, y_val) with the weights from epoch 150: loss 1.2082, accuracy 0.75. Training loss went 0.6964 -> 0.0194. Stopping at epoch 32 would have given val_loss 0.5272 and val_accuracy 0.775.
Simplified: A toy re-implementation in TypeScript, not Keras or TensorFlow: 2 input features, at most 3 hidden layers of up to 32 units, a seeded 2-D dataset (inside/outside a circle with label noise; 40 validation points), plain SGD without momentum, shuffle=False so batch order is fixed, float64 arithmetic, and binary cross-entropy computed from the output logit (same value as on probabilities, without clipping). The results match an independent NumPy implementation of the same procedure, not a real Keras run.
Loading the simulation…
Why Observe Training
When training begins, you usually cannot know in advance how many epochs will be needed to reach the best validation loss. One approach is to train long enough for overfitting to begin, choose a suitable number of epochs, and then start another training run from scratch. That approach wastes time. Keras callbacks provide a way to observe training as it happens and respond during the run.
A Keras callback is an object passed to fit that is called at various points during training.
A callback turns training from a process that only runs toward a fixed endpoint into a process that can observe training behavior and respond while the model is fitting.
Callback Invocation Timeline
The model invokes callbacks at points during training. The exact response depends on the callback and the training information it monitors. For example, a callback can be designed to respond to a selected training event, inspect the current training information, and then save progress, interrupt training, change the learning rate, or perform another action.
The important idea is not that every callback performs an action at every listed point. Instead, a callback is invoked at training events relevant to its implementation. Built-in callbacks use those opportunities to inspect a monitored metric and decide whether their configured response should occur.
Metrics and Patience
A callback needs a monitored metric so that it can relate its response to training behavior. Training loss and validation loss are examples of the behavior that can be observed during fitting. A callback can use the monitored metric to determine whether the model is improving or whether progress has stopped.
Patience controls how long a callback waits after the monitored metric stops improving before it responds. The response depends on the callback: the configured patience period can influence when training is interrupted, when the learning rate is changed, or when a callback takes another configured action.
Selecting a Built-In Callback
The main selection question is what you want to change in response to training behavior. ModelCheckpoint is for preserving model weights. EarlyStopping is for interrupting training when a monitored metric has stopped improving for the configured patience period. ReduceLROnPlateau is for changing the learning rate when validation loss has stopped improving instead of immediately stopping training.
| Callback | Primary response | Use it when |
|---|---|---|
| ModelCheckpoint | Saves weights during training | Preserving model progress matters |
| EarlyStopping | Interrupts training | Continuing is no longer useful after the configured patience period |
| ReduceLROnPlateau | Changes the learning rate | Validation loss has stopped improving and training should adjust rather than immediately stop |
Choose the callback by the action you want it to take.
Choosing by desired action
A training run needs three different forms of control: preserve the best model according to a monitored metric, stop when the metric no longer improves, and change the learning rate when validation loss stops improving.
- 1Preserve progress
Choose ModelCheckpoint because its purpose is to save weights during training and it can keep only the best model according to a monitored metric.
- 2End unproductive training
Choose EarlyStopping because it interrupts training when a monitored metric stops improving for the configured patience period.
- 3Adjust continued training
Choose ReduceLROnPlateau because it changes the learning rate when validation loss stops improving.
The correct sequence is ModelCheckpoint, EarlyStopping, and ReduceLROnPlateau. The decision comes from the intended response: save, stop, or change the learning rate.
Custom Training Responses
Built-in callbacks do not cover every possible training action. When the required behavior is not saving weights, interrupting training, or changing the learning rate in the ways provided by the built-in callbacks, a custom callback can provide the missing response.
A custom callback is created by subclassing keras.callbacks.Callback and implementing methods that run at selected training events.
Because the callback is connected to the training process, its event methods can respond when the selected events occur. A custom callback can also access model information, including information associated with the model, training logs, metrics, or optimizer, when that information is available to the callback during the event.
Think of a custom callback as a listener attached to training. The training loop reaches a selected event, the callback method is invoked, and the method uses the information available at that point to perform its custom behavior. The callback is therefore not a separate training run; it is a response mechanism inside the current run.
Mistakes Beginners Make
Common mistake Choosing a callback without identifying the desired response.
What it looks likeUsing EarlyStopping when the real goal is to preserve the best weights.
Why it failsEarlyStopping interrupts training, while ModelCheckpoint saves weights during training.
Do this insteadDecide first whether the response should be saving, stopping, or changing the learning rate.
Common mistake Treating patience as a metric.
What it looks likeDescribing patience as if it were training loss or validation loss.
Why it failsPatience is a configured waiting period used after the monitored metric stops improving.
Do this insteadSeparate the monitored metric from the amount of time the callback waits before responding.
Common mistake Assuming that a built-in callback covers every possible action.
What it looks likeTrying to force a built-in callback to perform a training action that is not among its supported purposes.
Why it failsBuilt-in callbacks do not cover every possible training action.
Do this insteadCreate a custom callback by subclassing keras.callbacks.Callback and implementing methods for selected training events.
Common mistake Ignoring validation behavior when choosing ReduceLROnPlateau.
What it looks likeChoosing it without considering whether validation loss has stopped improving.
Why it failsReduceLROnPlateau changes the learning rate when validation loss stops improving.
Do this insteadUse validation loss as the relevant behavior when the goal is to adjust the learning rate after a plateau.
Apply the Decision Rule
Choose ModelCheckpoint, EarlyStopping, ReduceLROnPlateau, or a custom callback for each situation. First, you want to keep the best model according to a monitored metric. Second, you want training to stop after a monitored metric fails to improve for the configured patience period. Third, validation loss has stopped improving, and you want to change the learning rate rather than immediately stop. Fourth, you need a training action not covered by the built-in callbacks.
Hints
- Match each situation to the intended action: save, stop, change the learning rate, or implement a new response.
- The monitored metric and patience wording identify the control behavior, but the desired action identifies the callback.
The answers are ModelCheckpoint, EarlyStopping, ReduceLROnPlateau, and a custom callback, respectively.
Key Takeaways
- A Keras callback is an object passed to fit and invoked at points during training.
- Callbacks allow training behavior to be observed and acted on without requiring a separate trial run.
- ModelCheckpoint preserves weights, EarlyStopping interrupts training, and ReduceLROnPlateau changes the learning rate when validation loss stops improving.
- The monitored metric describes the behavior being observed, while patience determines how long a callback waits before responding.
- Custom callbacks subclass keras.callbacks.Callback, implement methods for selected training events, and can access model information available during those events.