Tensor Data Types
Rank and shape describe structure. Dtype describes the kind of data contained in that structure.
Three Questions About One Tensor
When describing a tensor, ask three different questions. How many axes does it have? How large is it along each axis? What kind of data does it contain? These questions correspond to rank, shape, and data type, or dtype. Together, they describe both the tensor's structure and its contents.
Rank and shape describe structure. Dtype describes the kind of data contained in that structure.
Counting Axes as Rank
A tensor's rank is the number of axes in the tensor. In Python libraries such as NumPy, this quantity is also called ndim. Rank describes the tensor's structure, not the total number of values stored in it. A one-axis tensor has a different structure from a two-axis tensor, and a three-dimensional tensor has three axes.
Finding Rank from a Shape
A tensor has shape (3, 3, 5). What is its rank?
- 1Read the shape
The shape contains the three entries 3, 3, and 5.
- 2Count the entries
Counting the entries gives three, so the tensor has three axes.
- 3State the rank
The tensor's rank is 3. In a Python library such as NumPy, its ndim is also 3.
The tensor has rank 3, or ndim 3.
Reading a Shape Tuple
A tensor's shape records its size along each axis. It is written as a tuple of integers. The number of entries in the shape equals the tensor's rank. Therefore, a shape with two entries describes a two-axis tensor, while a shape with three entries describes a three-axis tensor.
| Tensor structure | Shape example | Rank |
|---|---|---|
| Scalar | () | 0 |
| Vector | (5,) | 1 |
| Matrix | (3, 5) | 2 |
| 3D tensor | (3, 3, 5) | 3 |
The number of entries in a shape gives the tensor's rank.
The parentheses and comma matter for special cases. A vector can have the single-entry shape (5,), while a scalar has the empty shape (). These shapes show that a vector has one axis and a scalar has no axes.
Identifying the Data Type
The dtype identifies the kind of data contained in a tensor. Examples include float32, uint8, and float64. Unlike rank and shape, dtype does not describe how many axes the tensor has or how large it is along those axes. It describes the tensor's contents.
Separating Shape from dtype
A tensor is described by shape (3, 5) and dtype float32. What does each attribute tell you?
- 1Interpret the shape
The shape has two entries, 3 and 5. This means the tensor has two axes, with sizes 3 and 5 along those axes.
- 2Find the rank
Because the shape has two entries, the tensor's rank is 2.
- 3Interpret the dtype
The dtype float32 identifies the kind of data contained in the tensor.
Shape (3, 5) describes structure; dtype float32 describes the contained data type.
Practical Interpretation
Read tensor descriptions in a fixed order. First count the shape entries to determine rank. Next read each shape entry as the size along its corresponding axis. Finally inspect dtype to determine the kind of data contained. This order keeps structural information separate from information about the tensor's values.
The source also identifies a rare char tensor as another possible data type. It notes that string tensors do not exist in NumPy, or in most other libraries, because tensors live in preallocated, contiguous memory segments and variable-length strings would prevent this implementation.
Common Interpretation Mistakes
Common mistake Treating shape as the data type
What it looks likeReading (3, 5) as if it identified what kind of values the tensor contains.
Why it failsThe shape records sizes along axes. The dtype identifies the contained data type.
Do this insteadRead (3, 5) as structural information and look separately for a dtype such as float32, uint8, or float64.
Common mistake Counting values instead of axes to find rank
What it looks likeConcluding that a tensor with shape (3, 5) has rank 15.
Why it failsRank counts the entries in the shape, which correspond to axes, not the total number of values.
Do this insteadCount the shape entries. The shape (3, 5) has two entries, so its rank is 2.
Common mistake Ignoring the empty shape of a scalar
What it looks likeAssuming every tensor must have at least one shape entry.
Why it failsA scalar has the empty shape ().
Do this insteadRecognize that the empty shape represents a tensor with no axes.
Common mistake Confusing a one-entry shape with an empty shape
What it looks likeTreating (5,) and () as equivalent.Why it failsThe shape (5,) describes a vector with one axis, while () describes a scalar with no axes.
Do this insteadCount the entries: (5,) has rank 1, and () has rank 0.
Check Your Understanding
A tensor has shape (3, 3, 5) and dtype uint8. Identify its rank, state the size along each axis, and explain what uint8 describes.
Hints
- Count the entries in the shape to find rank.
- Match each shape entry to one axis.
- dtype describes the kind of data contained.
Compare a vector with shape (5,) and a scalar with shape (). Which has an axis, and what is the rank of each?
Hints
- The number of shape entries gives rank.
- A one-entry shape and an empty shape represent different structures.
Key Takeaways
- 1A tensor is described by rank, shape, and dtype.
- 2Rank counts the tensor's axes and is also called ndim in Python libraries such as NumPy.
- 3Shape is a tuple of integers recording the size along each axis.
- 4The number of entries in a shape equals the tensor's rank.
- 5dtype identifies the kind of data contained and is separate from shape and rank.
At a glance
- Rank counts axes; it does not count the total number of tensor values.
- Shape records the size along each axis, and its number of entries gives rank.
- A vector can have shape (5,), a scalar has shape (), a matrix can have shape (3, 5), and a 3D tensor can have shape (3, 3, 5).
- dtype identifies the contained data type, such as float32, uint8, or float64.
- Shape and rank describe structure, while dtype describes contents.