The Learner in Statistical Learning

Training data is the learner’s input and consists of a finite sequence of labeled examples.

Programming

The Learner’s Input

Before a learner can use information in a learning problem, it needs input. In statistical learning, that input is training data: a finite sequence of labeled examples. Each example connects a domain point with the label associated with it.

Key point

The word labeled is essential. A collection of inputs becomes training data when each input is paired with a label.

Building the Sequence

How are multiple labeled examples arranged as a finite sequence given to the learner?

Training data is written as S = ((x₁, y₁), ..., (xₘ, yₘ)). This notation shows that the learner receives a finite sequence with m labeled examples. The parentheses group each input and its label into one example, while the outer sequence collects all examples into S.

Worked example

Reading a Training Sequence

Interpret S = ((x₁, y₁), (x₂, y₂), (x₃, y₃)) as training data.

1
Find the examples
The three parenthesized pairs are the individual training examples: (x₁, y₁), (x₂, y₂), and (x₃, y₃).
2
Separate each pair
In every pair, xᵢ is the input or domain point, and yᵢ is the label associated with that input.
3
Identify the complete data
The entire sequence is S, so S is the complete training data and can also be called the training set.

Result

The sequence contains three labeled training examples, and the complete collection is the training set S.

Inside One Labeled Example

What does each labeled training example contain, and how are the input and label connected?

Definition

A training-data pair is an element of X × Y. Its first component, xᵢ, is a domain point from X. Its second component, yᵢ, is the corresponding label from Y.

The pairing is what connects information about an example to the result associated with that example. The input alone describes the domain point, but the label supplies the associated outcome needed for labeled learning data.

In the real world

Consider papayas described by properties such as color and softness. Those properties provide information about each described papaya, while tastiness is the result associated with it. The collection becomes training data when each description is paired with its tastiness label.

Labels Versus Inputs

What is present in labeled training data that is missing from a collection containing only inputs?

Collection	What each item contains	Is it labeled training data?
Labeled training data	An input from X paired with a label from Y	Yes
Collection of inputs	Inputs without their associated labels	No

The presence of the input-label pairing distinguishes labeled training data from inputs alone.

From Examples to Training Set

How do individual training examples combine to form the training set and the complete training data sequence?

A training example is one labeled pair, such as (xᵢ, yᵢ). Training data is the complete finite sequence of those pairs. Training set is another name for the complete training data S, not the name of one individual pair.

One pair is one training example.
A finite sequence of pairs is the training data.
The complete training data S is also called the training set.

Common Terminology Mistakes

Common mistake
Calling an input by itself a labeled training example.
What it looks like
Treating xᵢ as the complete example.
Why it fails
A training example is a pair containing both an input and its label.
Do this instead
Write the complete example as (xᵢ, yᵢ).
Common mistake
Calling a collection of inputs training data without checking for labels.
What it looks like
Collecting domain points but keeping no associated y values.
Why it fails
Training data consists of labeled examples, not inputs alone.
Do this instead
Verify that every input is paired with its corresponding label.
Common mistake
Using training set to mean one example.
What it looks like
Referring to (x₁, y₁) as the training set.
Why it fails
The training set is another name for the complete training data S.
Do this instead
Use training example for one pair and training set for the full sequence.

Best practice

When reading a description of a learning problem, identify three levels separately: the input xᵢ, the labeled pair (xᵢ, yᵢ), and the complete sequence S. This prevents the terms example, training data, and training set from being mixed together.

Check Your Understanding

EASY

A learning problem contains a finite collection of domain points. For each point, a corresponding label is also recorded. Explain whether the collection is training data, identify the form of one training example, and state what the complete collection can be called.

Hints

Ask whether each domain point is paired with a label.
Represent one example as (xᵢ, yᵢ).
Use the term training set for the complete training data.

Your attempt (not saved)

What do you think happens?

Suppose a collection contains x₁, x₂, and x₃, but no y values. Is it already labeled training data?

Yes, because several inputs are present
No, because the inputs are not paired with labels

Reveal answer

Answer:

No, because the inputs are not paired with labels.

Training data requires a finite sequence of labeled examples, and each example contains an input paired with a label.

Key Takeaways

1
Training data is the learner’s input and is a finite sequence of labeled examples.
2
Each training example is a pair (xᵢ, yᵢ), with xᵢ from X and yᵢ from Y.
3
The complete sequence is written as S = ((x₁, y₁), ..., (xₘ, yₘ)).
4
Training set is another name for the complete training data S.
5
Inputs without associated labels form a collection of inputs, not labeled training data.

At a glance

Training data is the learner’s input: a finite sequence of labeled examples.
A labeled example contains an input from X and a label from Y.
The notation S = ((x₁, y₁), ..., (xₘ, yₘ)) represents the complete training data.
Individual pairs are training examples, while the complete collection is the training set.
A collection of inputs is not labeled training data until labels are paired with those inputs.

More concepts starting with T