Spike-Timing-Dependent Plasticity

STDP makes synaptic change sensitive to both the order and the relative timing of presynaptic and postsynaptic spikes.

Programming

Why Timing Matters

A synapse does not necessarily change simply because neural activity occurred. In spike-timing-dependent plasticity, or STDP, the relationship between two events matters: an incoming presynaptic spike and a later postsynaptic spike. The order of those events helps determine whether the synapse becomes stronger or weaker.

Key point

STDP makes synaptic change sensitive to both the order and the relative timing of presynaptic and postsynaptic spikes.

How does synaptic strength change when the presynaptic spike occurs before versus after the postsynaptic spike, and how does the size of the change vary with the time difference?

Reading the STDP Rule

Definition

Spike-timing-dependent plasticity is a Hebbian form of synaptic plasticity that adds a temporal condition: the relative timing of presynaptic and postsynaptic activity helps determine the direction of synaptic change.

Suppose a synapse carries an incoming spike toward a postsynaptic neuron. In the most studied form of STDP described in the source, the synapse is strengthened when the presynaptic spike arrives shortly before the postsynaptic neuron fires. The interpretation is that the incoming activity has the right temporal relationship to the postsynaptic event. If the order is reversed, so that the postsynaptic neuron fires first and the presynaptic spike arrives shortly afterward, the synapse is weakened. The size of the change also depends on the relative timing, not only on which event came first.

Worked example

Comparing Two Spike Orders

Compare two cases at the same synapse: in Case A, the presynaptic spike is closely followed by a postsynaptic spike; in Case B, the postsynaptic spike occurs first and the presynaptic spike follows shortly afterward.

1
Case A
The presynaptic event comes first and the postsynaptic event follows closely. In the most studied STDP form, this timing relationship produces an increase in synaptic strength.
2
Case B
The order is reversed. The postsynaptic event comes first, followed by the presynaptic event, so the synapse decreases in strength in the same STDP form.
3
Compare the intervals
The time difference between the two spikes also matters. STDP is therefore sensitive to both direction of order and relative timing.

Result

The same synapse can be strengthened or weakened depending on which spike comes first and how closely the two spikes are timed.

Adding the Third Factor

Reward-modulated STDP extends the timing rule with a third factor: neuromodulatory input. Presynaptic and postsynaptic spikes can create a potential synaptic change, but that potential does not become a lasting modification unless a suitable neuromodulatory signal arrives within a relevant time window afterward.

The three factors are presynaptic activity, postsynaptic activity, and neuromodulation. The first two establish the timing relationship. The third determines whether that potential change is consolidated as a lasting synaptic modification. In the dorsal striatum, the neuromodulatory factor described in the source is dopamine, and the relevant synaptic sites are the spines of medium spiny neurons.

How do presynaptic activity, postsynaptic activity, and a dopamine-like reward signal combine over time to determine whether a synapse is strengthened or weakened?

From Spikes to Eligibility

The timing sequence can be understood as a temporary record followed by a later evaluation. First, presynaptic and postsynaptic spikes occur in a relationship that STDP considers relevant. That relationship creates a potential synaptic modification, often described in the source as an eligibility trace. A later neuromodulatory signal can then convert that temporary eligibility into a lasting synaptic change if it arrives within the relevant time window.

What happens first when pre- and postsynaptic spikes occur, when is an eligibility trace formed, and when does a later dopamine signal convert it into lasting synaptic change?

What do you think happens?

A suitable presynaptic-postsynaptic timing pattern occurs, but no dopamine-like neuromodulatory signal arrives during the relevant later window. What should you expect under reward-modulated STDP?

The timing pattern automatically becomes a lasting synaptic change
The timing pattern creates a potential change, but lasting modification is not established
The synapse must become stronger because the presynaptic spike came first

Reveal answer

Answer:

The timing pattern creates a potential change, but lasting modification is not established.

Reward-modulated STDP requires the timing relationship and a later neuromodulatory factor. The source states that lasting change occurred only when the neuromodulatory pulse arrived within the relevant time window.

Actor and Critic Connections

The actor in an actor-critic system changes the tendencies that produce actions. STDP supplies one ingredient for actor-like change because it can select synapses according to their recent contribution to a postsynaptic firing event. Reward-modulated STDP adds outcome sensitivity: a later neuromodulatory signal can determine whether a timing-related synaptic change becomes lasting.

The connection to reinforcement learning is especially close in the actor unit of an actor-critic network. The source states that the actor unit's learning rule closely corresponds to reward-modulated STDP, and that the relevant biological discussion includes corticostriatal synapses. This is a correspondence between a learning rule and a biological mechanism; ordinary STDP and the actor unit are not identical.

How does a reward-modulated change at corticostriatal synapses influence the actor, while a critic-like system evaluates the outcome, and where does the biological analogy stop?

Evidence and Its Limits

The historical connection begins with A. H. Klopf's hedonistic neuron hypothesis, which suggested that neurons behave in a hedonistic manner by seeking to maximize their activity. The hypothesis influenced the actor-critic network of Barto and colleagues in 1983, whose neuron-like actor unit used a Law-of-Effect-style learning rule.

Claim	What the mechanism supports	What it does not establish
STDP	Synaptic change can depend on the order and relative timing of pre- and postsynaptic spikes.	STDP alone is not a complete actor-like reinforcement-learning rule.
Reward-modulated STDP	Timing-related synaptic changes can become lasting when suitable neuromodulation arrives later.	The mechanism alone does not prove that the brain implements a complete actor-critic algorithm.
Actor-unit correspondence	An actor unit's learning rule closely corresponds to reward-modulated STDP.	A correspondence does not make the computational rule and biological mechanism identical.
Klopf's hypothesis	The historical idea helped motivate early actor-critic models.	Not all details of the original hypothesis match current knowledge about synaptic plasticity.

Separate biological evidence, computational correspondence, and stronger claims about whole-brain algorithms.

Which parts of the actor-critic interpretation are directly supported by observed neural mechanisms, and which parts remain an analogy or unresolved inference?

Common mistake
Treating STDP as a rule that strengthens every active synapse.
What it looks like
A learner says that any synapse involved in neural activity must become stronger.
Why it fails
STDP depends on the relative timing and order of presynaptic and postsynaptic activity. Reversing the order can weaken the synapse.
Do this instead
Ask which spike occurred first and how closely the two spikes were timed.
Common mistake
Calling ordinary STDP a complete reward-learning rule.
What it looks like
A learner describes STDP as sufficient for an actor to learn from later outcomes.
Why it fails
The source states that actor-like outcome-sensitive learning also requires a neuromodulatory factor such as dopamine.
Do this instead
Distinguish two-factor timing conditions from reward-modulated STDP's additional neuromodulatory factor.
Common mistake
Assuming a suitable spike pattern guarantees lasting change under reward-modulated STDP.
What it looks like
A learner ignores whether dopamine arrives after the timing event.
Why it fails
The source reports that lasting corticostriatal changes required a neuromodulatory pulse within a relevant later time window.
Do this instead
Separate the temporary eligibility created by spike timing from the later signal that can make the change lasting.
Common mistake
Treating the actor-unit correspondence as proof of a biological actor-critic algorithm.
What it looks like
A learner concludes that observed reward-modulated STDP demonstrates that the brain implements actor-critic learning in every detail.
Why it fails
The source describes the evidence as supporting biological plausibility, not establishing a complete algorithmic implementation.
Do this instead
Use cautious language such as supports, corresponds to, or makes plausible.
Common mistake
Assuming every detail of the hedonistic neuron hypothesis remains accepted.
What it looks like
A learner treats Klopf's original proposal as a complete description of current synaptic plasticity.
Why it fails
The source states that not all details of the proposal match what has since been learned.
Do this instead
Treat the hypothesis as an important historical influence while separating it from modern evidence.

Apply the Distinction

MEDIUM

A corticostriatal synapse experiences a presynaptic spike shortly before a postsynaptic spike. Several seconds later, dopamine arrives within the relevant time window. Explain the role of each of the three factors and state why this observation supports, but does not prove, an actor-critic interpretation.

Hints

Identify the presynaptic and postsynaptic timing relationship first.
Explain what the timing creates before dopamine arrives.
Use the phrase supports rather than proves when discussing the actor-critic interpretation.

Your attempt (not saved)

Worked example

Evaluating a Biological Interpretation

Determine whether the following conclusion is justified: Because dopamine-dependent timing-sensitive changes occur at corticostriatal synapses, the brain has been proven to implement a complete actor-critic algorithm.

1
Identify the supported observation
The evidence supports the existence of a mechanism in which presynaptic and postsynaptic timing combines with later neuromodulatory input to produce lasting synaptic change.
2
Connect it to the actor
That mechanism provides a biologically plausible route for changing action-related synapses in an outcome-sensitive way, which is relevant to actor-like learning.
3
Check the strength of the conclusion
The source explicitly limits the inference. Evidence for reward-modulated STDP supports, but does not by itself prove, that some animals have something like an actor-critic system.

Result

The conclusion is too strong. The evidence supports biological plausibility and a close correspondence with actor-unit learning, but it does not prove a complete actor-critic implementation in every detail.

Key Takeaways

1
STDP makes synaptic change depend on the order and relative timing of presynaptic and postsynaptic spikes.
2
In the most studied STDP form described here, presynaptic activity shortly before postsynaptic firing strengthens a synapse, while the reversed order weakens it.
3
Reward-modulated STDP is a three-factor process: presynaptic activity, postsynaptic activity, and later neuromodulation such as dopamine.
4
Spike timing can create an eligibility trace, while a later neuromodulatory signal can help convert that potential change into lasting synaptic modification.
5
Reward-modulated STDP makes actor-like, outcome-sensitive synaptic learning biologically plausible, but it does not prove that the brain implements a complete actor-critic algorithm.

At a glance

STDP is sensitive to the order and relative timing of presynaptic and postsynaptic spikes.
Reward-modulated STDP adds a later neuromodulatory factor, such as dopamine, to the timing-based rule.
The timing event can create eligibility for change, while neuromodulation can help make the change lasting.
This mechanism corresponds closely to actor-unit learning and supports an actor-like biological interpretation.
The evidence establishes biological plausibility, not a complete proof that the brain implements actor-critic learning in every detail.

Related concepts

More concepts starting with S