DeepDream

DeepDream does not begin by inventing an image from nothing. It begins with an existing image and changes that image so that a convolutional network responds more strongly inside selected layers. This makes DeepDream a method for revealing and manipulating patterns the network has learned. The same source material also places DeepDream beside variational autoencoders, which provide a different generative perspective: learning a compressed latent representation from existing data and using that learned space as a basis for producing new images.

A variational autoencoder is called generative because its purpose is not merely to classify or recognize existing data. It learns statistical structure from examples and can use that structure to produce new data with related characteristics. For images, the visible pixels are the data available at the start. The model learns a compressed latent representation behind those pixels. Latent means that this structure is not directly visible in the original image.

DeepDream takes an existing image and uses a convolutional network to evaluate it. Instead of leaving the pixels unchanged after evaluation, it changes the pixels so that activations inside selected network layers become larger. As the process repeats, patterns recognized by the network become visually prominent. The output is therefore an altered version of the starting image, not an unrelated image created without a base.

DeepDream needs a numerical objective so that it can determine whether a change is making the chosen activations larger. Gradient ascent repeatedly changes the input pixels in the direction that increases this objective. In this process, the network evaluates the image, the objective provides the target, and the gradient identifies a direction for changing the image. Repeating the update turns the original image into a stronger expression of the selected layer's learned features.

The DeepDream objective is a weighted combination of activation magnitudes from selected layers. The source describes these contributions as L2-based magnitudes. Each layer has a coefficient that determines how strongly it contributes. The activation contributions are normalized by the number of activation values, and border pixels are left out in the cited formulation to help avoid border artifacts.

DeepDream also works at several image scales called octaves. Processing begins with a smaller version of the image and then moves through increasingly larger versions. Each successive scale is 1.4 times the previous scale, or 40 percent larger. Working across these scales allows feature maximization to act while the image is examined at different sizes, improving the quality of the visualization.

DeepDream follows the broad idea used in convolutional-network filter visualization: change the input in the direction that increases an internal response. The distinction is in both the target and the starting image. Filter visualization can maximize a specific filter in an upper layer and can begin with blank, slightly noisy input. DeepDream instead retains an existing image as its base and maximizes an entire layer. Because a whole layer combines many learned feature responses, the resulting image can contain a mixture of visual patterns rather than one isolated feature.

DeepDream depends on the features learned by the convolutional network used to evaluate the image. A pretrained network is therefore central to the result. The source example uses the Inception V3 model available with Keras, while also noting that other pretrained convolutional networks can be used. Since different architectures learn different features, changing the network can change the appearance of the visualization.