Concepts / Defining and Using Classes

Defining and Using Classes

The example does not require counting every training point. The central test is whether a point belongs to the subset that matters for representing the boundary, typically because it lies on the border between the classes.

  • Programming

The Boundary Question

When an SVM is trained to separate two classes, it does not treat every training data point as equally important for representing the decision boundary. Instead, it learns how important each point is to that boundary. The points that lie on the border between the classes are typically the most important. These points are called support vectors, which gives the support vector machine its name.

Training Data to Boundary

Training dataPoints from class 0 andclass 1Point importanceLearned during trainingSupport vectorsPoints on the class borderDecision boundarySeparates the two classesis evaluated foridentifies important pointshelps represent
How does information from the training data lead to a decision boundary separating the two classes?

The learning process can be understood as a change in focus. The SVM begins with training data from the two classes. During training, it learns the importance of the individual points for representing the boundary. The points that lie on the border between the classes become the support vectors, and these support vectors help define the decision boundary.

Support Vectors on the Border

Definition

Support vectors are the subset of training data points that matter for representing the decision boundary, typically because they lie on the border between the two classes.

Class 0Training pointsSupport vector 0On the class borderDecision boundaryBetween the classesSupport vector 1On the class borderClass 1Training pointsincludeshelps representhelps representincludes
Which training data points determine the boundary between the two classes, and how are the support vectors positioned relative to that boundary?

The important relationship is not simply that a support vector belongs to one class. Its importance comes from its position at the border between the classes. Support vectors from the two classes are associated with the boundary that separates those classes.

Relevant and Non-Relevant Points

Support vectorsTypically on the classborderBoundaryrepresentationImportant to the boundaryOther trainingpointsNot typically on the classborderLower boundaryrelevanceTypically do not define theboundaryhelps representtypically has
What is the difference between training points that become support vectors and points that do not affect the learned decision boundary?
Training-point groupPosition described in the sourceRole in representing the boundary
Support vectorsTypically on the border between the classesMatter for defining or representing the decision boundary
Other training pointsNot identified as the border pointsTypically do not matter as much for representing the decision boundary

The distinction is based on importance to the decision boundary, especially whether a point lies on the border between the classes.

A training set can contain many points, but typically only a subset matters for representing the decision boundary. Calling a point a support vector does not mean that it is a separate class. It means that the point is one of the important training points associated with the class border.

A Small Boundary Example

Worked example

Identifying the support vectors

Imagine training data containing points from class 0 and class 1. A few points lie on the border between the two classes, while the remaining points are farther from that border. Which points would typically be the support vectors?

  1. 1
    Step 1: Identify the two classes

    Separate the training data conceptually into class 0 and class 1, the two classes that the SVM must distinguish.

  2. 2
    Step 2: Locate the class border

    Focus on the points that lie on the border between the two classes. Their position makes them important for representing the boundary.

  3. 3
    Step 3: Name the important points

    The border points are the support vectors. The other training points are not typically the points that define the decision boundary.

Result

The support vectors are the subset of training points lying on the border between class 0 and class 1.

Key point

The example does not require counting every training point. The central test is whether a point belongs to the subset that matters for representing the boundary, typically because it lies on the border between the classes.

Common Identification Errors

  • Common mistake

    Treating every training point as equally important to the decision boundary.

    What it looks like

    Assuming that all points contribute in the same way because they all appear in the training data.

    Why it fails

    The SVM learns how important each training point is, and typically only a subset matters for representing the boundary.

    Do this instead

    Look first for the points on the border between the classes; these are the points typically called support vectors.

  • Common mistake

    Defining a support vector as a third class.

    What it looks like

    Thinking that class 0, class 1, and support vectors are three separate categories.

    Why it fails

    Support vector describes the role of a training point in representing the boundary, not a new class.

    Do this instead

    Keep the two classes separate from the support-vector role. A support vector is a point from the training data associated with the class border.

  • Common mistake

    Ignoring the point’s position relative to the class border.

    What it looks like

    Calling a point a support vector without considering whether it lies on the border between the classes.

    Why it fails

    The source identifies the border between the classes as the typical location of the points that matter for defining the decision boundary.

    Do this instead

    Use the point’s relationship to the border as the main clue when identifying support vectors.

Check Your Understanding

EASY

An SVM is trained with data from two classes. Some points lie on the border between the classes, and many other points do not. Explain which points are typically called support vectors and why they matter.

Hints
  • Start with the relationship between a point and the border between the classes.
  • Then connect that relationship to the decision boundary.

What do you think happens?

If a training point lies on the border between class 0 and class 1, is it typically more likely to be important for representing the decision boundary than a point that is not on the border?

  • Yes

  • No

Reveal answer
Answer:

Yes

The points that typically matter for defining the decision boundary are the ones that lie on the border between the classes. These points are called support vectors.

Key Takeaways

  1. 1
    An SVM learns from training data belonging to two classes.
  2. 2
    During training, it learns how important each training point is for representing the decision boundary.
  3. 3
    Typically, the points on the border between the classes are the important points.
  4. 4
    These points are called support vectors.
  5. 5
    Support vectors give the support vector machine its name because they help define the boundary between the two classes.

At a glance

  • An SVM separates two classes by learning from labeled training data.
  • Only a subset of training points typically matters for representing the decision boundary.
  • Points on the border between the classes are called support vectors.
  • Support vectors are important because they help define the boundary separating the two classes.

Related Topics

Related concepts

More concepts starting with D