Measure Concentration

Many later arguments take a vector as input and then measure or compare it.

Programming

One Vector, Several Measurements

Many later arguments take a vector as input and then measure or compare it. The vector itself does not change when we choose a measurement rule. What changes is the feature we ask the rule to emphasize: geometric length, the total magnitude across coordinates, or the single largest coordinate magnitude.

In the real world

Use the vector u = (2, −1, 3) as a running example. Three norms inspect this same vector differently: the Euclidean norm combines squared coordinate magnitudes and takes a square root, the ℓ1 norm adds coordinate magnitudes, and the ℓ∞ norm keeps only the largest coordinate magnitude.

Column Coordinates in R^d

The setting is finite-dimensional Euclidean space. A vector with d coordinates belongs to R^{^d}. In this chapter, vectors are written as column vectors, so their coordinates are arranged vertically rather than across a row. The symbol d records how many coordinates the vector has; it does not change the fact that the vector is represented as one vertical object.

How are the coordinates arranged in a column vector, and how does that arrangement distinguish a vector from its transpose?

Combining Two Vectors

For two vectors u and v in R^{^d}, their inner product is written as ⟨u, v⟩. It combines corresponding coordinates: multiply the first coordinates, multiply the second coordinates, and continue through coordinate d. Adding those coordinatewise products produces one scalar. The inner product therefore connects two vectors rather than measuring only one vector by itself.

Formula

How do corresponding coordinates from two vectors combine, and how do their products become one scalar?

Worked example

An inner product calculation

Calculate the inner product of u = (2, −1, 3) and v = (1, 4, 2).

1
Match coordinates
Pair the first coordinates, the second coordinates, and the third coordinates.
2
Multiply pairs
The coordinate products are 2·1, (−1)·4, and 3·2.
3
Add the products
The inner product is 2 − 4 + 6.

Result

⟨u, v⟩ = 4

Euclidean Length from Self-Combination

When the two inputs to the inner product are the same vector, the result contains the squared coordinate magnitudes. Taking the square root converts that quantity into the Euclidean norm, also called the ℓ2 norm. Thus, the inner product supplies the quantity inside the square root.

Formula, step by step

1
Use the same vector twice: ⟨u, u⟩.
2
This produces the sum of squared coordinate magnitudes.
3
Take the square root to obtain the Euclidean norm.

How does taking the inner product of a vector with itself produce the squared Euclidean length, and what changes when the square root is taken?

Worked example

Euclidean norm of u

Calculate the Euclidean norm of u = (2, −1, 3) from its inner product with itself.

1
Form the self-inner-product
⟨u, u⟩ = 2² + (−1)² + 3².
2
Evaluate the squared magnitudes
The sum is 4 + 1 + 9 = 14.
3
Take the square root
The Euclidean norm is √14.

Result

∥u∥₂ = √14

Three Ways to Measure u

The ℓ2, ℓ1, and ℓ∞ norms use different rules on the same coordinates. The ℓ2 norm reflects overall geometric length by combining squared coordinate magnitudes and taking a square root. The ℓ1 norm totals the magnitudes of all coordinates. The ℓ∞ norm keeps only the largest coordinate magnitude.

What different feature of a vector does each norm emphasize: overall geometric length, total coordinate magnitude, or the largest coordinate?

Formula

How do the absolute values of all coordinates contribute to the ℓ1 norm, and how is the largest absolute coordinate selected for the ℓ∞ norm?

Worked example

The ℓ1 and ℓ∞ norms of u

Calculate the ℓ1 norm and ℓ∞ norm of u = (2, −1, 3).

1
Find coordinate magnitudes
The absolute values are |2| = 2, |−1| = 1, and |3| = 3.
2
Calculate ℓ1
Add every coordinate magnitude: 2 + 1 + 3 = 6.
3
Calculate ℓ∞
Keep only the largest coordinate magnitude: max{2, 1, 3} = 3.

Result

∥u∥₁ = 6 and ∥u∥∞ = 3

A Reliable Calculation Order

1
First identify the requested measurement: inner product, ℓ2 norm, ℓ1 norm, or ℓ∞ norm.
2
For an inner product, pair corresponding coordinates, multiply each pair, and add the products.
3
For the ℓ2 norm, use the vector with itself in the inner product and then take the square root.
4
For the ℓ1 norm, take the absolute value of every coordinate and add all those magnitudes.
5
For the ℓ∞ norm, take the absolute value of every coordinate and select the largest magnitude.
6
Before finalizing the result, check whether the rule uses all coordinates, squared magnitudes, or only one largest magnitude.

Common mistake
Using the largest coordinate itself for the ℓ∞ norm.
What it looks like
For u = (2, −1, 3), choosing −1 because it is the numerically smallest coordinate.
Why it fails
The ℓ∞ norm uses the largest coordinate magnitude, not the coordinate with a particular sign.
Do this instead
Compare 2, 1, and 3 after taking absolute values. The result is 3.
Common mistake
Adding signed coordinates for the ℓ1 norm.
What it looks like
Computing 2 + (−1) + 3 instead of adding magnitudes.
Why it fails
The ℓ1 norm totals coordinate magnitudes.
Do this instead
```
Compute |2| + |−1| + |3| = 6.
```
Common mistake
Stopping at the self-inner-product when the question asks for the Euclidean norm.
What it looks like
```
Reporting 14 for u = (2, −1, 3).
```
Why it fails
The self-inner-product supplies the quantity inside the square root.
Do this instead
Take the square root after calculating ⟨u, u⟩, giving ∥u∥₂ = √14.
Common mistake
Treating the three norms as measurements of three different vectors.
What it looks like
Assuming u changes when moving from ℓ2 to ℓ1.
Why it fails
The vector remains the same; only the measurement rule changes.
Do this instead
Keep the coordinates fixed and apply the requested norm definition.

Practice Check

EASY

Let w = (−2, 0, 4). Calculate ⟨w, w⟩, ∥w∥₂, ∥w∥₁, and ∥w∥∞. Then state which feature of w each norm emphasizes.

Hints

For ⟨w, w⟩, square each coordinate and add the results.
For ∥w∥₂, take the square root of ⟨w, w⟩.
For ∥w∥₁ and ∥w∥∞, use the absolute coordinate values 2, 0, and 4.

Your attempt (not saved)

Worked example

Practice check solution

For w = (−2, 0, 4), calculate the self-inner-product and the three norms.

1
Self-inner-product
⟨w, w⟩ = (−2)² + 0² + 4² = 20.
2
Euclidean norm
∥w∥₂ = √20.
3
ℓ1 norm
∥w∥₁ = |−2| + |0| + |4| = 6.
4
ℓ∞ norm
∥w∥∞ = max{2, 0, 4} = 4.

Result

⟨w, w⟩ = 20, ∥w∥₂ = √20, ∥w∥₁ = 6, and ∥w∥∞ = 4.

Key Takeaways

1
Vectors in this chapter are column vectors in finite-dimensional Euclidean spaces, and a vector with d coordinates belongs to R^{^d}.
2
The inner product ⟨u, v⟩ combines corresponding coordinate products from two vectors into one scalar.
3
The Euclidean norm is obtained by taking the square root of a vector's inner product with itself: ∥u∥₂ = √⟨u, u⟩.
4
The ℓ1 norm adds all coordinate magnitudes, while the ℓ∞ norm keeps only the largest coordinate magnitude.
5
The same vector can receive different measurements because each norm emphasizes a different feature.

At a glance

Vectors in this chapter are vertical column vectors in R^{^d}.
The inner product combines corresponding coordinates from two vectors and produces one scalar.
The Euclidean norm uses the square root of the self-inner-product.
The ℓ1 norm measures total coordinate magnitude, while the ℓ∞ norm measures the largest coordinate magnitude.
Different norms measure different features of the same unchanged vector.

More concepts starting with M

One Vector, Several Measurements

Column Coordinates in R^d

Combining Two Vectors

Euclidean Length from Self-Combination

Three Ways to Measure u

A Reliable Calculation Order

Practice Check

Key Takeaways

At a glance

Related Topics