Weight Vectors and Bias Terms

A halfspace hypothesis assigns one of two labels to an input vector.

Programming

From Features to Two Labels

Suppose every instance in a feature space must receive one of two labels. A halfspace hypothesis provides the rule for making that binary decision. It separates the feature space into regions associated with the labels -1 and +1. The separation is controlled by a weight vector and a bias term.

How does a halfspace hypothesis map an input vector to one of two class labels?

The Separating Boundary

The weight vector and bias parameter together determine the halfspace hypothesis. In two dimensions, the boundary associated with that hypothesis is a hyperplane that appears as a line. This line separates the feature space into two regions. The weight vector is perpendicular to the line, so it provides the direction used to describe the relationship between the boundary and the feature space.

How do the weight vector and bias term define a line that divides two-dimensional feature space?

The word halfspace refers to the regions created by the separating hyperplane. The hyperplane itself is the boundary; the regions on its sides are the parts of feature space associated with the two possible classifications.

Parameter Roles

Part of the hypothesis	Role described in the model
Weight vector	Helps determine the hypothesis and is perpendicular to the hyperplane in two dimensions.
Bias term	Works with the weight vector to determine the hypothesis.
Hyperplane	The separating boundary; in two dimensions, it is a line.
Regions on the sides	The areas associated with the positive and negative labels.

The weight vector and bias term jointly determine the halfspace hypothesis.

How do the weight vector and bias term contribute to the hypothesis?

Reading the Two Sides

Worked example

Classifying Points Relative to a Line

Imagine a two-dimensional feature space divided by the line produced by a halfspace hypothesis. Determine the label for an instance described as above the line and for an instance described as below the line.

1
Locate the boundary
The hyperplane is the separating line in the two-dimensional feature space.
2
Locate the first instance
An instance described as above the hyperplane belongs to the region associated with the positive label.
3
Locate the second instance
An instance described as below the hyperplane belongs to the region associated with the negative label.
4
Interpret the geometry
The words above and below describe the geometric relationship with the hyperplane and the direction of the weight vector. They do not necessarily mean physical up and down in an application.

Result

The first instance receives the positive label, and the second instance receives the negative label.

How does the position of an input point relative to the hyperplane determine whether it receives the positive or negative label?

Common Interpretation Errors

Common mistake
Treating the hyperplane as one of the two regions.
What it looks like
Calling the separating line itself the positive halfspace.
Why it fails
The hyperplane is the boundary. The regions on its sides are the parts associated with the two classifications.
Do this instead
Distinguish the boundary from the two regions it separates.
Common mistake
Assuming above and below always mean physical up and down.
What it looks like
Interpreting a positive label as meaning that an instance is physically higher.
Why it fails
The terms describe the geometric relationship with the hyperplane and the direction of the weight vector.
Do this instead
Use the model's geometry rather than a literal physical interpretation.
Common mistake
Forgetting that the hypothesis has two possible labels.
What it looks like
Describing a halfspace hypothesis as if it assigned many unrelated categories.
Why it fails
A halfspace hypothesis assigns one of two labels to an input vector.
Do this instead
Keep the binary outcome in view: the two labels are -1 and +1.
Common mistake
Treating the weight vector or bias term as the entire hypothesis by itself.
What it looks like
Explaining the classification rule using only the weight vector while ignoring the bias term.
Why it fails
The weight vector and bias parameter together determine the hypothesis.
Do this instead
Describe both parameters as parts of the same halfspace hypothesis.

Check Your Understanding

MEDIUM

A two-dimensional feature space contains a line that separates instances into two regions. Explain what the line represents, what determines the corresponding hypothesis, and how an instance on each side receives its label.

Hints

Identify the geometric name for the separating line.
Mention both the weight vector and the bias term.
Use the positive and negative labels -1 and +1.

Your attempt (not saved)

1
A halfspace hypothesis assigns one of two labels to an input vector. Its weight vector and bias term jointly determine the hypothesis. In two dimensions, the associated hyperplane is a line perpendicular to the weight vector. The line divides feature space into two regions: instances on the positive side receive the positive label, while instances on the negative side receive the negative label.

Key Takeaways

A halfspace hypothesis is a binary classification rule for an input vector.
The weight vector and bias term together determine the hypothesis.
In two-dimensional feature space, the hyperplane is a line perpendicular to the weight vector.
The hyperplane separates feature space into regions associated with positive and negative labels.
Above and below describe geometric relationships with the boundary and the weight-vector direction, not necessarily physical directions.

More concepts starting with W