Concepts / Variance of a Random Variable

Variance of a Random Variable

Variance measures expected squared distance from the mean, while Chebyshev's Inequality uses that variance to bound the probability of a large deviation.

  • Programming
Interactive lab

Try it: Expectation and Variance of a Random Variable

A discrete random variable X as values with probabilities: the expectation E[X] = Σ x·p(x) is a probability-weighted average, and the variance is the expected squared distance from that mean, computed two ways.

How it works

  1. Each outcome has a value x and a bar height; probabilities are the heights renormalised, p(x) = height / total, so they always sum to exactly 1.
  2. E[X] = μ is built term by term: every outcome adds x·p(x) to the running sum.
  3. Var[X] = E[(X − μ)²]: centre each value on μ, square the deviation, weight it by p(x) and add.
  4. The same variance again as E[X²] − μ², and the standard deviation √Var[X].
  5. All sums are exact fractions (shown alongside decimals).

Default run (22 steps): X has 6 outcomes with bar heights 10, 10, 10, 10, 10, 10 (total W = 60). … E[X] = μ = 7/2 = 3.5; Var[X] = 35/12 ≈ 2.9167; standard deviation = √Var ≈ 1.7078.

Simplified: Up to 8 outcomes with integer values from −20 to 20 and integer bar heights 0–100. If every bar is 0 the first outcome gets height 1 (a distribution needs total probability 1).

Educational simulation

Loading the simulation…

Interactive lab

Try it: Concentration: Hoeffding, Chebyshev and Markov

The sample mean of m i.i.d. bounded samples concentrates around the true mean μ: repeated seeded experiments estimate P(|sample mean − μ| ≥ ε) and compare it with Hoeffding's bound 2·exp(−2mε²), Chebyshev's bound Var/ε² and Markov's bound for the one-sided event.

How it works

  1. Draw θ1, …, θm i.i.d. Bernoulli(p) (bounded in [a, b] = [0, 1], common mean μ = p) and track the running sample mean.
  2. The deviation event is |sample mean − μ| ≥ ε for a chosen positive tolerance ε.
  3. Hoeffding: P(|mean − μ| ≥ ε) ≤ 2·exp(−2mε²/(b − a)²); Chebyshev with Var[mean] = p(1 − p)/m: ≤ p(1 − p)/(mε²); Markov for the non-negative mean: P(mean ≥ μ + ε) ≤ μ/(μ + ε).
  4. Repeat the whole experiment many times (seeded) and count how often each event happens.
  5. Compare the observed frequencies with the bounds: they are upper bounds, not the exact probability.

Default run (32 steps): θ1, …, θm are i.i.d. Bernoulli(0.5) samples, each in [a, b] = [0, 1], with true mean μ = 0.5. m = 100, ε = 0.1, seed 7. … 200 experiments: |mean − μ| ≥ 0.1 happened 3 times (frequency 0.015) — Hoeffding allows at most 0.2707, Chebyshev 0.25. One-sided mean ≥ μ + ε: 1 times (0.005) vs Markov 0.8333.

Simplified: Samples are Bernoulli(p) (0 or 1), so [a, b] = [0, 1]; the randomness comes from a seeded pseudo-random generator (mulberry32). Observed frequencies are estimates from a finite number of experiments.

Educational simulation

Loading the simulation…

Why Spread Matters

A random variable does not always equal its mean. To describe how widely its values can spread, we use variance. Chebyshev's Inequality then connects that spread to a probability statement: it gives an upper bound on the chance that the random variable is at least a chosen distance away from its mean.

Key point

Variance measures expected squared distance from the mean, while Chebyshev's Inequality uses that variance to bound the probability of a large deviation.

From Values to Variance

The construction of variance follows three steps. First, compare the random variable Z with its mean E[Z]. This produces the deviation Z − E[Z]. Next, square the deviation. Squaring makes the deviation nonnegative and gives more weight to larger distances. Finally, take the expected value of the squared deviation. The result is Var[Z], the variance of Z.

Formula, step by step
  1. 1
    Find the mean E[Z].
  2. 2
    Center the random variable by forming Z − E[Z].
  3. 3
    Square the centered deviation to obtain (Z − E[Z])².
  4. 4
    Take the expected value of the squared deviation.
Zpossible valueZ − E[Z]deviation from mean(Z − E[Z])²squared deviationVar[Z]expected squared deviationsubtract E[Z]squaretake expectation
How does each value's distance from the mean become the expected squared deviation?

Chebyshev's Bound

Definition

Chebyshev's Inequality states that the probability that Z differs from its mean by at least a chosen distance a is at most the variance divided by the squared distance: P(|Z − E[Z]| ≥ a) ≤ Var[Z] / a².

Read the inequality from left to right. The left side describes the deviation event: Z is at least a distance a from its mean. The right side is the bound that can be calculated from two inputs: the variance in the numerator and the squared deviation threshold in the denominator.

Z ≤ E[Z] − aE[Z] − a < Z < E[Z]+ aVar[Z] / a²upper boundZ ≥ E[Z] + apart of |Z − E[Z]| ≥ apart of |Z − E[Z]| ≥ a
How do the mean-centered interval and its tail regions correspond to the probability bound?

Applying the Calculation

Worked example

A numerical Chebyshev bound

Suppose a random variable has variance 4. Find an upper bound for the probability that it differs from its mean by at least 2.

  1. 1
    Identify the inputs

    The variance is 4 and the deviation threshold is a = 2.

  2. 2
    Substitute into Chebyshev's Inequality

    Use P(|Z − E[Z]| ≥ a) ≤ Var[Z] / a², giving P(|Z − E[Z]| ≥ 2) ≤ 4 / 2².

  3. 3
    Evaluate the denominator

    The squared threshold is 2² = 4.

  4. 4
    Compute the bound

    The right-hand side is 4 / 4 = 1.

Result

The probability that Z is at least 2 units from its mean is bounded above by 1.

Var[Z]variancea²squared thresholdVar[Z] / a²calculated boundP(|Z − E[Z]| ≥ a)bounded probabilityadeviation distancesquarenumeratordenominatorupper-bounds
Given a variance and a deviation distance, what calculation produces the upper bound?

Changing the Bound

Quantity that changesPosition in the boundEffect described by the formula
Variance Var[Z]NumeratorA larger variance produces a larger value of Var[Z] / a².
Deviation threshold aDenominator after squaringA larger threshold produces a larger a², making Var[Z] / a² smaller when the variance is fixed.

Chebyshev's bound is Var[Z] divided by the squared deviation threshold.

The formula makes the roles of spread and distance visible. Variance appears in the numerator, so increasing variance increases the calculated bound when the threshold stays fixed. The threshold is squared in the denominator, so increasing the allowed distance from the mean increases the denominator and decreases the bound when variance stays fixed.

EASY

A random variable has variance 9. Use Chebyshev's Inequality to find an upper bound for the probability that it differs from its mean by at least 3.

Hints
  • Use P(|Z − E[Z]| ≥ a) ≤ Var[Z] / a².
  • Substitute Var[Z] = 9 and a = 3.

Averages of i.i.d. Variables

The source also considers the average of m independent and identically distributed random variables. In this setting, the variance of the average decreases as m increases. When the common variance is written as σ², the variance of the average is σ² divided by m. Thus, increasing the number of variables makes the variance term used in a Chebyshev bound smaller.

Rule

m = 1variance σ²m = 2variance σ² / 2m = 3variance σ² / 3m variablesvariance σ² / mincrease mincrease mcontinue
As the number of i.i.d. random variables increases, how does the variance of their average change?
Worked example

Comparing averages

Compare the variance of an average formed from 2 i.i.d. variables with the variance of an average formed from 8 i.i.d. variables, assuming the common variance is σ².

  1. 1
    Use the average-variance formula

    For m i.i.d. variables, the variance of the average is σ² / m.

  2. 2
    Substitute m = 2

    The first average has variance σ² / 2.

  3. 3
    Substitute m = 8

    The second average has variance σ² / 8.

  4. 4
    Compare the results

    The denominator is larger for 8 variables, so σ² / 8 is smaller than σ² / 2.

Result

The average based on 8 i.i.d. variables has the smaller variance.

Common Calculation Mistakes

  • Common mistake

    Using the deviation threshold without squaring it.

    What it looks like
    Writing Var[Z] / a instead of Var[Z] / a².
    Why it fails

    Chebyshev's bound uses the squared deviation threshold in the denominator.

    Do this instead

    Square a before dividing the variance by it.

  • Common mistake

    Treating variance as an ordinary distance from the mean.

    What it looks like
    Describing Var[Z] as Z − E[Z].
    Why it fails

    Variance is the expected value of the squared deviation, not the unsquared deviation itself.

    Do this instead
    Use Var[Z] = E[(Z − E[Z])²].
  • Common mistake

    Reading the Chebyshev result as the exact deviation probability.

    What it looks like
    Saying P(|Z − E[Z]| ≥ a) equals Var[Z] / a².
    Why it fails

    The inequality provides an upper bound.

    Do this instead

    Use the language at most or no greater than for the right-hand side.

  • Common mistake

    Forgetting that the average's variance depends on the number of variables.

    What it looks like

    Using σ² for every average regardless of m.

    Why it fails

    For an average of m i.i.d. variables, the variance is σ² / m.

    Do this instead

    Include m in the denominator.

Practice Check

What do you think happens?

A random variable has variance 16. Which expression is the Chebyshev upper bound for the probability that it differs from its mean by at least 4?

  • 16 / 4

  • 16 / 4²

  • 4 / 16²

  • 16 × 4²

Reveal answer
Answer:

16 / 4²

Chebyshev's Inequality places the variance in the numerator and the squared deviation threshold in the denominator.

MEDIUM

Explain in one or two sentences why increasing m makes the variance of the average σ² / m smaller.

Hints
  • Look at the position of m in the expression σ² / m.
  • Connect the smaller variance to the variance term in Chebyshev's bound.

Key Takeaways

  1. 1
    Variance is the expected squared distance from a random variable's mean: Var[Z] = E[(Z − E[Z])²].
  2. 2
    Chebyshev's Inequality states that P(|Z − E[Z]| ≥ a) ≤ Var[Z] / a².
  3. 3
    A larger variance produces a larger Chebyshev bound when the deviation threshold is fixed.
  4. 4
    A larger deviation threshold produces a smaller bound because the threshold is squared in the denominator.
  5. 5
    For an average of m i.i.d. random variables with common variance σ², the variance is σ² / m, so the variance decreases as m increases.

At a glance

  • Variance measures expected squared distance from the mean.
  • Chebyshev's Inequality converts variance and a deviation threshold into an upper bound on deviation probability.
  • The bound is Var[Z] divided by the squared threshold.
  • The variance of an average of m i.i.d. random variables is σ² / m and decreases as m increases.

Related Topics

Related concepts

More concepts starting with V