Convnet Filter Visualization

The central operation is gradient ascent on the convnet's input.

Programming

The Input Becomes the Object

Many convnet explanations focus on what happens inside the network. DeepDream reverses the usual direction of attention: instead of changing the convnet to fit an image, it changes the input supplied to the convnet. The input is the object being optimized, and the goal is to make a chosen filter more strongly activated.

Key point

The central operation is gradient ascent on the convnet's input.

What contains the optimized values, and how do those values move from the modified input through the convnet to the filter activation?

One Optimization Step

A filter-visualization process starts with an input and a selected filter. The convnet evaluates that input, and the selected filter has an activation. Gradient ascent then provides the direction for adjusting the input. The adjusted input is supplied to the convnet again, with the aim of increasing that filter's activation.

How do the filter activation, gradient, and updated input connect during each optimization step?

Worked example

Maximizing a Chosen Filter

Suppose a learner selects one filter and wants to visualize the input that activates it most strongly.

1
Select the objective
Choose the activation of one specific filter as the objective.
2
Evaluate the input
Supply the current input to the convnet and observe the selected filter's activation.
3
Use gradient ascent
Use the gradient-ascent direction to adjust the convnet input toward a stronger activation.
4
Repeat the process
Supply the adjusted input to the convnet again and continue optimizing the input toward the selected filter's objective.

Result

The optimized input is being used to visualize what the selected filter responds to, because the input has been adjusted to maximize that filter's activation.

Gradient Ascent Over Time

The important state change is not a change to the convnet itself. The input changes from one optimization step to the next. At each step, the current input is passed through the convnet, the selected filter's activation supplies the objective being maximized, and gradient ascent determines how to adjust the input for the next step.

How does the input image change step by step when gradient ascent increases a chosen filter's activation?

DeepDream and Filter Visualization

DeepDream and convnet filter visualization should not be treated as unrelated procedures. The source describes DeepDream as almost identical to the filter-visualization technique. Both optimize the convnet input with gradient ascent so that a selected filter's activation is maximized.

Aspect	DeepDream	Convnet filter visualization
What is changed	The input supplied to the convnet	The input supplied to the convnet
Central operation	Gradient ascent on the input	Gradient ascent on the input
Objective	Maximize the activation of a selected filter	Maximize the activation of a selected filter
Relationship	Almost identical to filter visualization	The technique whose central idea DeepDream uses

The available source describes the two methods as nearly the same and does not specify their few simple differences.

What is the difference between optimizing a whole input image in DeepDream and visualizing the input preferred by one convnet filter?

What the Process Reveals

The purpose of the process is to visualize a convnet filter. Repeatedly modifying the input to amplify a chosen filter reveals an input that the convnet has been optimized to associate with a stronger activation of that filter. DeepDream can therefore be understood as an image-directed use of a convnet: the image changes so that the selected filter responds more strongly.

What does DeepDream reveal by repeatedly modifying the convnet's input to amplify features it detects?

Mistakes Beginners Make

Common mistake
Assuming the convnet is modified during DeepDream.
What it looks like
Treating the procedure as if the network's parameters were changed to fit the image.
Why it fails
The source states that DeepDream changes the input supplied to the convnet and uses that input as the object being optimized.
Do this instead
Track the input as the changing object; the selected filter's activation is the objective.
Common mistake
Describing gradient ascent as an adjustment to the filter.
What it looks like
Saying that gradient ascent changes the selected filter so it activates more strongly.
Why it fails
Gradient ascent provides the direction for adjusting the convnet input.
Do this instead
Say that the input is adjusted in a direction intended to maximize the selected filter's activation.
Common mistake
Treating DeepDream and filter visualization as completely different methods.
What it looks like
Explaining them as unrelated algorithms with unrelated objectives.
Why it fails
The source describes DeepDream as almost identical to convnet filter visualization.
Do this instead
Start with their shared central idea: optimize the convnet input with gradient ascent to maximize a selected filter's activation.
Common mistake
Adding an unsupported distinction between the two techniques.
What it looks like
Claiming a particular difference that is not stated in the available material.
Why it fails
The source mentions a few simple differences but does not specify them.
Do this instead
Report only the shared mechanism and acknowledge that the unspecified differences should not be filled in by guesswork.

Check the Mechanism

MEDIUM

A learner says: The goal of DeepDream is to train the convnet until it recognizes the starting image. Correct the statement using the roles of the input, gradient ascent, and selected filter activation.

Hints

Identify which object is optimized.
Identify what gradient ascent changes.
Identify the objective that is maximized.

Your attempt (not saved)

Worked example

A Complete Trace

Trace the roles in a filter-visualization step from the current input to the next input.

1
Current input
The current input is supplied to the convnet.
2
Selected activation
The activation of the chosen filter is treated as the objective.
3
Gradient-ascent direction
Gradient ascent provides the direction for adjusting the input.
4
Next input
The adjusted input is supplied again so the selected filter's activation can be optimized further.

Result

The input changes step by step while the convnet is used to guide those changes toward a larger activation of the selected filter.

Key Takeaways

1
DeepDream uses the convnet input as the object being optimized.
2
Its objective is to maximize the activation of a specific filter.
3
Gradient ascent supplies the direction for adjusting the input.
4
Convnet filter visualization and DeepDream share the same central mechanism and are described as almost identical.
5
The process visualizes a filter by repeatedly modifying the input used by the convnet.

At a glance

DeepDream optimizes the input rather than changing the convnet.
Gradient ascent adjusts the input toward stronger activation of a chosen filter.
Filter visualization and DeepDream have the same central idea in the available material.
Tracing the process means following the input, filter activation, gradient direction, and updated input.
The optimized input provides a visualization of what strongly activates the selected filter.

Related concepts

More concepts starting with C