Models of Classical Conditioning

Classical conditioning is not only a matter of recording which events occur together. A useful model must also explain how learning changes from one trial to the next. Prior conditioning can alter what an animal learns later, and the time at which information becomes available can affect how predictions change during a conditioning episode.

The word reinforcement has not always had one fixed meaning in animal learning. According to the source, the term first appeared, to the best of its knowledge, in the 1927 English translation of Pavlov's monograph. In that use, reinforcement referred to a non-action-contingent case: the event was not described as depending on an action.

Later traditions used the term differently. Mackintosh proposed using reinforcement for either strengthening or weakening a pattern of behavior. Skinner used reinforcement only for strengthening behavior and treated weakening as something produced by punishment. These differences are historically important because the same word can point to different ideas depending on the theoretical tradition.

Kamin blocking shows that prior conditioning can affect subsequent learning. The situation has two stages. First, one stimulus is repeatedly involved in conditioning. Later, that previously conditioned stimulus appears together with a new stimulus. The question is whether the new stimulus will also become a predictor of the outcome. In blocking, prior conditioning prevents or greatly reduces the expected new learning about the added stimulus.

This idea gives blocking a learning-based explanation. Before the compound stage, Cue A has already acquired predictive significance. When Cue A and Cue B appear together, the outcome is not as unexpected as it would have been for a learner with no prior conditioning. Since the outcome is already prepared for, the new Cue B may contribute little or no additional learning.

The Rescorla-Wagner model is one model among several used to study classical conditioning. The source lists models associated with Klopf, Grossberg, Mackintosh, Moore and Stickney, Pearce and Hall, and Courville, Daw, and Touretzky. This broader list matters because it places Rescorla and Wagner within a history of animal learning theory rather than treating their model as the only available explanation.

Imagine following one conditioning episode from its beginning to its end. At the beginning, the learner has one set of expectations. As the episode unfolds, later events provide new information. The TD perspective compares these successive points rather than treating the whole episode as one undivided event. Learning is therefore connected to changes in predictive information over time.

The TD model developed within a larger history of learning rules. Sutton and Barto's 1981 work recognized a near identity between the Rescorla-Wagner model and the Least-Mean-Square learning rule, also called the LMS or Widrow-Hoff rule. Widrow and Hoff introduced the LMS rule in 1960. This connection places the TD model alongside an established error-based learning-rule tradition rather than presenting it as an unrelated account of conditioning.

The development was gradual. The early version of the TD model made a specific prediction: temporal primacy overrides blocking. The source reports that Kehoe, Scheurs, and Graham later showed this effect in the rabbit nictitating membrane preparation. This is an example of a model being evaluated against an experimental result, not merely a restatement that timing exists.

The TD model does not depend only on its learning rule. The way a stimulus is represented can also affect how the model performs. Later work examined different stimulus representations and their possible neural implementations in relation to response timing and response topography.

A temporal model needs a way to encode stimulus information across the episode. If the representation preserves useful distinctions between early and later moments, the model has a richer basis for comparing successive predictions. If those distinctions are not represented, the model has less information with which to organize time-sensitive changes.

Ludvig, Sutton, and Kehoe introduced the microstimulus representation in 2008. In 2012, they evaluated the TD model on previously unexplored classical-conditioning tasks and examined the influence of several stimulus representations, including the microstimulus representation. The important lesson is that evaluating a temporal learning rule also requires asking how the stimulus is represented.