Associative Strength in Conditioning Models

Classical conditioning is not simply produced by presenting two events together. The order and timing of the events matter. In the usual arrangement, a previously neutral stimulus, called the conditioned stimulus or CS, begins before the unconditioned stimulus or US. After learning, the conditioned response usually begins before the US arrives. This arrangement makes the CS useful as a prediction of what is coming next.

The temporal-difference model, or TD model, treats temporal differences in prediction as learning signals. At one moment, a stimulus produces a prediction about a later outcome. At a later moment, the system has another prediction, influenced by what has occurred in between. Learning depends on the difference between the prediction at the current state and the value backed up from a later state.

This gives the model a way to represent changing associative value. A stimulus does not need to be judged only by whether the US eventually appears. Its value can change as the trial progresses. A later stimulus may already have learned value, and that value can create a learning difference for an earlier stimulus.

Facilitation of remote associations occurs when a CS is followed by an empty trace interval and then by the US. Conditioning to the first stimulus can improve when a second CS occupies part of that otherwise empty interval. The second stimulus does more than add another event to the trial: it fills part of the temporal path between the first CS and the US.

The Egger-Miller effect shows that a closer temporal relationship with the US does not guarantee stronger conditioning to a stimulus. In the relevant arrangement, CSA and CSB overlap. CSB is temporally better placed relative to the US, yet adding CSA reduces conditioning to CSB compared with an arrangement in which CSA is absent.

Blocking normally occurs when one CS has already been trained to predict the US. When a second stimulus is added during later pairings, much less learning occurs about that new stimulus. The pretrained stimulus already accounts for the outcome, so the added stimulus receives little new associative strength in the usual arrangement.

The TD model predicts reversal of blocking when the newly added stimulus is moved earlier so that it begins before the pretrained stimulus. In that arrangement, the new stimulus can acquire associative strength. As its strength grows, the strength associated with the previously trained stimulus decreases.

Second-order conditioning extends prediction beyond direct CS–US pairings. First, CSB is trained with the US. Later, CSA is paired with CSB without the US. CSA can nevertheless acquire associative strength, even though CSA is never directly paired with the US.

The TD explanation is that CSB already has learned value. When the trial moves from CSA to CSB, the later CSB state can provide a nonzero learning difference for CSA. In the second phase, CSA first gains substantial strength while the strength associated with CSB decreases. This pattern has also been reported in animal experiments.