Neuronal Plasticity

A neuron may participate in an action before the consequence of that action becomes available as a learning signal. An eligibility trace solves this timing problem by preserving synapse-specific information about earlier activity. When a later reinforcing neuromodulatory signal arrives, it can use that record to modify the efficacy of a synapse that is still eligible.

An eligibility trace records that a particular synapse participated in earlier activity involving the neuron. The trace belongs to the synapse, not merely to the neuron as a whole. That distinction matters because different synapses may have participated in different activity. A later reinforcing signal can therefore modify the efficacy of an eligible synapse rather than indiscriminately changing every synapse on the neuron.

The sequence has two separated moments. First, activity involving a synapse is recorded as eligibility. Later, a reinforcing neuromodulatory input arrives. If the synapse is eligible at that later moment, the signal can modify the synapse's efficacy. The trace therefore acts as a bridge between the earlier activity and the later learning signal.

Think of the trace as carrying information across a delay. Activity involving a synapse produces an eligibility record. The system may then wait for the consequence associated with the neuron's activity. If the reinforcing neuromodulatory signal arrives while the synapse remains eligible, the record can be used to modify synaptic efficacy. The source describes the role of the trace in preserving earlier activity and connecting it to a later signal; it does not specify a particular numerical decay rule or time constant.

Actor and critic learning rules both use eligibility traces, but their traces carry different information. A contingent actor trace depends on the actor unit's input and its output. A non-contingent critic trace does not include the critic unit's output. The difference concerns what information is included in the trace, not whether the synapse has an activity record.

Klopf's hedonistic-neuron hypothesis provides an intuition for why an eligibility trace is useful. In this view, a neuron participates in a closed loop: its activity can influence later input through feedback within the nervous system and body or through the external environment. The neuron adjusts synaptic efficacies according to rewarding or punishing consequences associated with its action potentials. A synapse becomes eligible when it participated in the neuron's firing, and a reinforcing signal selects that eligible synapse for modification.

The neural interpretation becomes more specific in corticostriatal synapses. These synapses carry signals from the cortex to principal neurons in dorsal and ventral subdivisions of the striatum, and they also receive signals from dopamine neurons. Eligibility traces provide the bridge between earlier activity at a corticostriatal synapse and later neuromodulatory input relevant to changing its efficacy.