Data Representation in Deep Learning
In this data representation, an index on axis 0 identifies a sample because axis 0 is the samples axis.
From Dataset to Sample
A deep learning data tensor often contains multiple samples. In the MNIST example, the samples are images of digits. The complete tensor represents the available collection, but many operations need to work with one particular image or with a selected group of images. Tensor slicing provides the selection mechanism.
Tensor slicing is the selection of specific elements in a tensor. When the first axis is the samples axis, selecting along that axis identifies which sample or samples are being selected.
The First Axis as a Sample Selector
Axis 0 is the samples axis in the data tensors described here. This means that the first position in a tensor index identifies a position among complete samples. In the MNIST example, applying the index i to train_images selects one digit image. The key idea is that i is not simply selecting an unexplained position: it is selecting a sample because the first axis has the samples-axis role.
A sample is one individual data item in the tensor. In the MNIST example, one sample is one digit image.
A data batch is a group of samples considered together. Those samples occupy multiple positions along the samples axis.
Tracing a Selection
Selecting One Digit Image
Interpret the selection train_images[i] when train_images contains multiple digit images.
- 1Start
Begin with the complete
train_imagesdata tensor, which contains multiple samples. - 2Apply the index
Apply i along the tensor's first axis. Because the first axis is the samples axis, the index identifies a sample position.
- 3Reach the result
The selection arrives at one complete sample: one digit image.
train_images[i] selects one digit image from the tensor.
The same mental model applies when the selection identifies a group rather than one position. For example, the generated notation tensor[2:5] represents a selection of positions 2 through 4 along axis 0. That selection refers to a group of samples, so it can be understood as a data batch rather than as one sample. The important point is that slicing specifies a selection; it does not automatically mean the entire dataset.
Reading Tensor Selections Correctly
When interpreting a tensor selection, trace it in three parts: begin with the complete data tensor, identify the axis receiving the index or slice, and determine whether the result is one sample or a group of samples. For the MNIST expression train_images[i], the first axis is axis 0, axis 0 is the samples axis, and the result is one digit image.
Common mistake Treating train_images[i] as an arbitrary element with no data meaning.
What it looks likeReading i only as a bracket position.
Why it failsThe index is applied along the first axis, and that axis is the samples axis.
Do this insteadInterpret the indexed position as identifying one complete digit-image sample.
Common mistake Confusing one sample with a data batch.
What it looks likeCalling a selection of multiple samples one sample.
Why it failsA sample is one individual data item, while a batch is a group of samples considered together.
Do this insteadAsk whether the selection identifies one position on axis 0 or multiple positions on axis 0.
Common mistake Assuming every slice means the entire dataset.
What it looks likeTreating any selection from a tensor as the complete collection.
Why it failsSlicing means that a particular selection is being made.
Do this insteadTrace which elements or sample positions the selection identifies.
Practice the Mental Trace
Suppose a data tensor contains multiple digit-image samples. Without executing anything, explain what is selected by tensor[0] and what is selected by tensor[2:5] when axis 0 is the samples axis.
Hints
- Start by identifying the axis receiving the selection.
- Remember that axis 0 identifies sample positions.
- Decide whether each selection identifies one position or a group of positions.
What do you think happens?
What kind of result should you expect when selecting one index along axis 0 of a tensor whose axis 0 is the samples axis?
Reveal answer
One complete sample
A single index selects one position along the samples axis. In the MNIST example, that position identifies one digit image.
Key Takeaways
- 1Tensor slicing selects specific elements from a tensor.
- 2Axis 0 is the samples axis in the data representation described here.
- 3Selecting one index along axis 0 identifies one complete sample, such as one MNIST digit image.
- 4Selecting multiple positions along axis 0 identifies a group of samples.
- 5A sample is one data item, while a data batch is a group of samples considered together.
At a glance
- Tensor slicing is the selection of specific elements in a tensor.
- Axis 0 serves as the samples axis in the described data tensors.
- An index such as
train_images[i] selects one complete digit-image sample. - A range selection along axis 0 can identify a group of samples, which forms a data batch.
- To interpret a selection, trace the complete tensor, the selected axis, and the resulting sample or group.