Clustering
Clustering is an unsupervised grouping task.
Try it: Linkage-Based Clustering
How agglomerative clustering starts from singletons and repeatedly merges the two closest clusters under single (min), complete (max) or average linkage, building a dendrogram until a stopping rule (k clusters or distance threshold r) says stop.
How it works
- Start with every point in its own cluster and compute the distance matrix.
- Cluster distance: single = closest pair of members (minimum), complete = farthest pair (max linkage), average = mean over all pairs.
- Find the closest pair of current clusters (ties go to the pair listed first) and merge them; the cluster count drops by one.
- Update the new cluster's row of the distance matrix and record the merge height in the dendrogram.
- Stop when k clusters remain, or when every between-cluster distance is larger than r.
Default run (12 steps): 8 points, each its own cluster. Single linkage; stop when 3 clusters remain. … Done: 3 clusters — {A,B,C} {D,E,F,G} {H}.
Simplified: Toy 2-D data (3–10 points) with Euclidean distance. Real hierarchical clustering uses many more points, other distances and faster data structures.
Loading the simulation…
From Objects to Groups
Suppose you receive a collection of objects but no group labels. A natural question is whether some objects resemble one another more than they resemble the rest. Clustering addresses this question by organizing objects into groups so that similar objects are together and dissimilar objects are separated.
Clustering is an unsupervised grouping task. Unsupervised means that the starting collection does not come with known group labels for the model to reproduce. Instead, the model uses choices about similarity or distance to propose a grouping. The result is useful for exploring data and discovering meaningful structure.
The Clustering Model
A clustering model takes a collection of objects together with a way to judge resemblance or distance, then produces a proposed organization into groups. Its output is group membership or a grouping of the objects, rather than a set of supplied class labels.
The input is not just an abstract list of objects. The model needs a basis for deciding which objects or clusters are close and which are far apart. That choice matters because the resulting groups depend on the similarity and distance choices used by the clustering method.
A Boundary Around PCA
The supplied material does not define PCA, variance maximization, principal components, or dimensionality reduction. Therefore, those technical ideas cannot be established from this article's source material. What the material does establish is clustering: organizing objects into groups according to resemblance and distance.
For that reason, clustering and PCA should not be presented as the same task in this lesson. Clustering is supported here as a grouping task whose output is group membership. The supplied material does not provide enough information to define PCA or to make a complete technical comparison between the two methods. The correct conclusion is a boundary of knowledge: explain clustering from the available evidence, and do not invent a definition of PCA.
Merging the Closest Clusters
Linkage-based clustering begins with individual objects treated as separate clusters. It repeatedly identifies the closest pair of clusters and merges them. After every merge, the number of clusters is smaller than it was before. The process therefore changes the organization of the collection step by step, without changing the objects themselves.
A Linkage Trace
Trace a linkage-based procedure for five objects: P, Q, R, S, and T. The procedure repeatedly merges the closest pair of clusters.
- 1Start
The initial clusters are {P}, {Q}, {R}, {S}, and {T}. There are five clusters.
- 2First merge
If {P} and {Q} are the closest pair, merge them to form {P,Q}. The clusters are now {P,Q}, {R}, {S}, and {T}.
- 3Second merge
If {R} and {S} are the closest pair at the next round, merge them to form {R,S}. The clusters are now {P,Q}, {R,S}, and {T}.
- 4Interpretation
The important change is the organization of the collection. The objects remain P, Q, R, S, and T, while their proposed group membership changes.
This trace places P with Q, R with S, and T separately at the shown stage. A complete algorithm also needs a rule for when to stop merging.
Single Linkage Distance
Single Linkage defines the distance between two clusters as the minimum distance between any two members, with one member taken from each cluster. In plain language, two clusters are considered close if at least one cross-cluster pair is close.
This definition lets the algorithm compare clusters even after earlier merges have created groups containing multiple objects. The comparison searches across all pairs formed by taking one member from the first cluster and one member from the second, then uses the closest pair as the cluster distance.
The phrase closest clusters is incomplete until the algorithm states what cluster distance means. Single Linkage supplies one answer: use the closest pair of members across the two clusters. Another cluster-distance choice could produce a different sequence of merges.
Applications and Interpretation
Clustering can be applied to different kinds of objects because its central purpose is to discover meaningful groups. Computational biologists can group genes according to similarities in their expression across experiments. Retailers can group customers using customer profiles to support targeted marketing. Astronomers can group stars according to spatial proximity.
| Domain | Objects | Basis described in the source | Purpose |
|---|---|---|---|
| Biology | Genes | Similarities in expression across experiments | Identify meaningful groups |
| Retail | Customers | Customer profiles | Support targeted marketing |
| Astronomy | Stars | Spatial proximity | Identify meaningful groups |
Examples of clustering applications from the supplied material
These examples use different objects and different observations, but they share the same exploratory pattern: compare objects using a chosen notion of resemblance, organize them into groups, and inspect whether the groups help reveal structure that matters in the domain.
Where the Intuition Breaks
The phrase similar points belong together is a useful starting intuition, but it is not a complete specification. It does not say exactly how similarity should be measured for every data set. It also does not determine which cluster-distance rule to use or when the merging process should stop. As a result, more than one grouping may seem reasonable, and different algorithm choices can produce different clusterings.
Common mistake Treating clustering as if it reproduced known labels.
What it looks likeDescribing the groups as correct classifications before explaining how the groups were formed.
Why it failsThe source presents clustering as an unsupervised grouping task used to explore data when group labels are not supplied.
Do this insteadDescribe the result as a proposed grouping based on similarity or distance.
Common mistake Assuming that closest clusters has one universal meaning.
What it looks likeTracing a merge without stating how the distance between two multi-member clusters is calculated.
Why it failsA linkage-based method must define the distance between clusters, and different choices can lead to different clusterings.
Do this insteadState the linkage rule. For Single Linkage, use the minimum distance between any two members, one from each cluster.
Common mistake Assuming that clustering changes the objects.
What it looks likeSaying that P or Q has been transformed when the algorithm merges {P} and {Q}.
Why it failsThe important change is in how the collection is organized, not in the objects themselves.
Do this insteadDescribe the change as a change in proposed group membership.
Common mistake Inventing a technical definition of PCA from this material.
What it looks likeClaiming that the supplied source defines principal components or variance maximization.
Why it failsThe source explicitly does not define PCA, variance maximization, principal components, or dimensionality reduction.
Do this insteadState the evidence boundary and focus the explanation on the clustering concepts that are supported.
Check Your Reasoning
A clustering procedure has produced the groups {P,Q}, {R,S}, and {T}. Explain what this says about the current organization of the objects. Then identify two additional decisions that must be specified before the linkage-based procedure is fully defined.
Hints
- The groups describe membership, not a transformation of P, Q, R, S, or T.
- One missing decision concerns the distance between clusters.
- The other missing decision concerns when merging stops.
Practice Answer
Interpret the grouping {P,Q}, {R,S}, and {T}.
- 1Membership
P and Q are currently placed in one group, R and S in another, and T remains separate.
- 2Distance rule
The procedure must specify how to calculate the distance between two clusters. Single Linkage would use the closest pair of members across the two clusters.
- 3Stopping rule
The procedure must specify when to stop merging, because different stopping choices can leave different numbers of groups.
The grouping is a proposed organization based on resemblance or distance, not a claim that the objects themselves have changed.
Key Takeaways
- 1Clustering organizes objects into groups based on resemblance, similarity, or distance.
- 2A clustering model receives objects and a basis for comparison, then proposes group membership.
- 3Linkage-based clustering repeatedly merges the closest clusters, reducing the number of clusters at each round.
- 4Single Linkage defines cluster distance using the minimum distance between any two members from the two clusters.
- 5The supplied material does not define PCA or dimensionality reduction, so PCA should not be treated as the same established task as clustering in this lesson.
At a glance
- Clustering is an unsupervised way to organize objects into groups using similarity or distance.
- The output of a clustering model is a proposed grouping or group membership.
- Linkage-based clustering repeatedly merges the closest clusters, but it requires both a cluster-distance rule and a stopping rule.
- Single Linkage uses the closest pair of members across two clusters to define their distance.
- The supplied material establishes clustering but does not define PCA, variance maximization, principal components, or dimensionality reduction.