Pretrained ImageNet Models

DeepDream does not begin by creating an image from nothing. It starts with an existing image and changes the image so that a convolutional network responds more strongly inside selected layers. As the pixels are repeatedly adjusted, patterns learned by the network become visually prominent. The result is an altered version of the starting image rather than an unrelated collection of new pixels.

The network evaluates the current image and produces activations in its layers. DeepDream uses those activations to define an objective. Gradient ascent then determines how the input pixels should change so that the objective becomes larger. Repeating the adjustment makes the image a progressively stronger expression of features learned by the network.

The important direction is from the objective back to the input pixels. DeepDream is not merely recording what the network sees. It uses the response to decide how to alter what the network will see next. Because the process is repeated, small pixel changes accumulate into visible distortions connected to patterns already available in the original image.

DeepDream and filter visualization use the same broad strategy: change the input in the direction that increases an internal network response. Their distinctive choices are different. Filter visualization can maximize one specific filter, while DeepDream maximizes an entire layer. Since a whole layer combines many learned feature responses, the resulting image can contain a mixture of visual patterns instead of one isolated feature.

Gradient ascent needs one numerical objective to increase. DeepDream defines this objective as a weighted sum of L2-based activation contributions from selected layers. Each layer has a coefficient, so its contribution can have more or less influence on the total objective. The cited formulation normalizes contributions by the number of activation values and leaves out border pixels when calculating the activation contribution, helping avoid border artifacts.

You can interpret the loss as a collection of layer measurements being combined into one score. First, activation values from a selected layer contribute an L2-based magnitude. That contribution is normalized over the activation values, with border positions excluded in the cited formulation. Then the layer's weight determines how strongly that contribution affects the total. Gradient ascent changes the image to make this combined score larger.

DeepDream does not process the image at only one size. It defines several image scales called octaves. Processing begins with a smaller version and moves through increasingly larger versions. Each successive scale is 1.4 times the previous scale, which is 40 percent larger. Working across these scales allows feature maximization to act while the image is examined at different sizes, improving the quality of the visualization.

DeepDream depends on the features learned by the convolutional network that evaluates the image. The source example uses the Inception V3 model available with Keras, while also noting that other pretrained convolutional networks can be used. Because different architectures learn different features, changing the pretrained network can change the appearance of the resulting visualization.