NumPy arrays
A tensor is a container for data, almost always numerical data.
From Numbers to Tensors
When a machine-learning system works with numerical data, it needs a way to hold that data in an organized form. A tensor provides that container. The most useful way to approach a tensor is not as a mysterious new mathematical object, but as a broader category that includes matrices and extends the same organizing idea to any number of dimensions.
A tensor is a container for data, almost always numerical data.
Dimensions and Axes
A tensor's dimensions describe how its numerical data is organized. Tensors generalize matrices to an arbitrary number of dimensions, so a matrix is one particular case: a matrix is a two-dimensional tensor. In tensor terminology, a dimension is often called an axis.
A Worked Shape Example
Adding dimensions to numerical data
Compare three generated arrangements of numerical data and identify what changes as dimensions are added.
- 1One dimension
Consider the generated sequence [2, 4, 6]. It is organized along one dimension, so it can be described as a one-dimensional tensor.
- 2Two dimensions
Now organize generated values into rows: [[2, 4, 6], [8, 10, 12]]. This is a matrix, and therefore it is a two-dimensional tensor.
- 3Three dimensions
Add another layer of rows: [[[2, 4, 6], [8, 10, 12]], [[14, 16, 18], [20, 22, 24]]]. This arrangement has three dimensions, so it is a three-dimensional tensor.
- 4Name the dimensions
Each dimension in the three-dimensional arrangement can be referred to as an axis. The values are still numerical data; the added dimensions describe how that data is organized.
The key change is the number of dimensions used to organize the values. A matrix is already a tensor; adding dimensions produces tensors with more axes.
This example illustrates the generalization rather than a special category of values. The numerical contents can vary, while the tensor idea remains the same: data is held in a container and organized across one or more dimensions.
Why Machine Learning Uses Tensors
Machine-learning systems need a way to hold the numerical data they work with. Tensors provide that kind of container, which makes them the basic data structure of current machine-learning systems. Their importance is practical: machine learning relies on tensors to represent the numerical data its systems use.
Tensors are not merely an optional way to organize numbers. They are central because machine-learning systems rely on them to represent the numerical data those systems use.
Common Conceptual Mistakes
Common mistake Treating a tensor as something completely separate from a matrix.
What it looks likeThinking that a matrix is not a tensor because tensors sound more advanced.
Why it failsA matrix is a two-dimensional tensor.
Do this insteadTreat a matrix as one specific member of the broader tensor category.
Common mistake Confusing an axis with a new kind of numerical value.
What it looks likeAssuming that an axis is an additional value stored alongside the data.
Why it failsIn tensor terminology, a dimension is often called an axis.
Do this insteadUse axis as the name for one dimension of the tensor's organization.
Common mistake Thinking that tensors are important only because they have many dimensions.
What it looks likeFocusing on high-dimensional arrangements while ignoring their role as containers for numerical data.
Why it failsThe practical importance of tensors comes from their role as the basic data structure of current machine-learning systems.
Do this insteadRemember both parts: a tensor organizes data, almost always numerical data, and machine-learning systems rely on it to represent the numerical data they use.
Practice Check
Explain the relationship among a matrix, a tensor, a dimension, and an axis in your own words. Then describe what changes when a two-dimensional numerical arrangement becomes a three-dimensional one.
Hints
- Start with the fact that a matrix has two dimensions.
- Use the tensor definition as a container for numerical data.
- Remember that a dimension is often called an axis.
What do you think happens?
A matrix is two-dimensional. Is it also a tensor?
Reveal answer
Yes
A matrix is a two-dimensional tensor. Tensors generalize matrices to an arbitrary number of dimensions.
Essential Takeaways
- 1A tensor is a container for data, almost always numerical data.
- 2Tensors generalize matrices to an arbitrary number of dimensions.
- 3A matrix is a two-dimensional tensor.
- 4A tensor dimension is often called an axis.
- 5Tensors are central to current machine-learning systems because those systems rely on them to represent the numerical data they use.
At a glance
- A tensor is a container for numerical data.
- Matrices are two-dimensional tensors, while tensors can have an arbitrary number of dimensions.
- Each tensor dimension is often called an axis.
- Tensors are the basic data structure used by current machine-learning systems to represent numerical data.