Feature Vectors
The important conceptual trace is: object in X, feature representation for that object, and possible label from Y. The feature vector describes the domain point; the label set supplies the allowed label values.
Two Questions Before Learning
A statistical learning problem begins by separating two questions: What objects are we studying, and what labels could describe those objects? The collection of objects is called the domain set and is written as X. The collection of possible labels is called the label set and is written as Y.
Consider a learning problem about papayas. The domain set X contains all papayas. The label set Y is {0, 1}, so the possible labels are 0 and 1. A papaya is an object being studied; 0 and 1 are possible labels for that object.
From Papaya to Features
An individual instance in X is called a domain point. A domain point is usually represented by a feature vector. For a papaya, color and softness are features. These features provide a representation of the papaya for the learning problem. They describe the domain point; they do not become labels.
A feature vector is a vector whose positions represent the features used to represent a domain point. In the papaya example, the relevant features are color and softness.
The Label Set
The label set Y is the collection of possible labels that may describe domain objects. In the papaya example, Y is {0, 1}. The values 0 and 1 are labels, while color and softness are features of a domain point.
| Collection | Notation | Contains in the papaya example | Role |
|---|---|---|---|
| Domain set | X | All papayas | Objects being studied |
| Label set | Y | 0 and 1 | Possible labels |
| Feature representation | Feature vector | Color and softness | Description of a domain point |
The domain set, label set, and feature representation answer different questions.
Considering an Object and Label
Tracing One Papaya
Show how one papaya is handled in the learning setup.
- 1Select the domain point
Begin with one particular papaya. It is an individual instance in X, so it belongs on the domain side of the problem.
- 2Represent the domain point
Represent that papaya using a feature vector whose features include color and softness.
- 3Consider a possible label
The possible labels come from Y, which is {0, 1} in this example. A possible label is therefore 0 or 1, not the papaya and not one of its features.
- 4Form the labeled consideration
The domain point and a possible label are considered together as x from X and y from Y. The feature vector describes x, while y supplies the allowed label value.
The object remains a domain point in X, its representation uses features, and the possible label comes separately from Y.
The important conceptual trace is: object in X, feature representation for that object, and possible label from Y. The feature vector describes the domain point; the label set supplies the allowed label values.
Mistakes About X and Y
Common mistake Putting a papaya in the label set
What it looks likeTreating one particular papaya as one of the possible labels
Why it failsA papaya is an object being studied, so it belongs on the domain side in X.
Do this insteadPlace all papayas, including one particular papaya, in X. Place the possible label values in Y.
Common mistake Treating color and softness as labels
What it looks likeCalling color and softness the two labels in the papaya problem
Why it failsColor and softness are features used to represent a domain point. The labels in the example are 0 and 1.
Do this insteadUse color and softness in the feature representation, and use 0 or 1 as a possible label from Y.
Common mistake Confusing a feature vector with the label set
What it looks likeAssuming that the vector representation itself is Y
Why it failsThe feature vector describes a domain point, while Y is the collection of possible labels.
Do this insteadKeep the representation of x separate from the possible label y.
Check the Separation
For the papaya learning problem, classify each item as belonging to X, Y, or the feature representation: all papayas, one particular papaya, 0, 1, color, and softness. Then describe the path from one papaya to a possible labeled example.
Hints
- Start by asking whether the item is an object, a possible label, or a feature.
- Remember that X contains the papayas and Y contains 0 and 1.
- The path should include selecting a domain point, representing it with features, and considering a possible label from Y.
- 1X is the domain set: the collection of objects being studied. An individual object in X is a domain point. A domain point is usually represented by a feature vector; in the papaya example, color and softness are features. Y is the label set: the collection of possible labels. For papayas, Y is {0, 1}. A domain point and a possible label can be considered together, but the object, its features, and its label remain distinct concepts.
Key Takeaways
- The domain set X contains the objects being studied.
- An individual object in X is a domain point and is usually represented by a feature vector.
- Color and softness are features of a papaya domain point, not labels.
- The label set Y contains possible labels; for the papaya example, Y is {0, 1}.
- A domain object x and a possible label y can be considered together while remaining distinct roles.