23 Jul 2026This page is about another data science technique, the k-means clustering algorithm. Where a classifier such as the k-nearest neighbours algorithm is handed neatly labelled examples and learns to sort new ones into those classes, clustering is given no labels at all. Its job is to look at a plain cloud of points and discover, entirely on its own, which of them naturally belong together.
The points above are penguins — one dot per bird — measured on two features of their bill: its length along the top ridge, the culmen, and its depth. That is exactly the situation k-means finds itself in: a scatter of measurements with the labels stripped away. Squint at the plot and you can already sense a few loose groups; the algorithm's task is to turn that hunch into a clean verdict.
Before k-means can find groups it must be told how many to look for. That number is the parameter k, and here we have set it to five. This is the first and most important way k-means parts company with k-nearest neighbours: in kNN the k counts how many neighbours get to vote on a single new point, whereas in k-means it fixes how many clusters the whole data set will be carved into. The algorithm begins by dropping k markers — the black crosses, called centroids — onto five randomly chosen penguins, then paints every point with the colour of whichever centroid is nearest. The result is a first, rough partition entirely at the mercy of where those flags happened to land.
A random start is rarely a good one, so k-means improves it by repeating two simple steps. First each centroid slides to the average position — the mean — of all the points currently wearing its colour; this is the "means" in k-means. Then the algorithm assigns new cluster membership by choosing the closest center as the only indicator. Press play to watch the crosses wander from their random starting spots and settle into the heart of each group, the colours shifting with them until nothing more changes. This tug-of-war between points and centres is known as Lloyd's algorithm. Unlike kNN — which never really trains, but simply memorises the data and answers questions on demand — k-means iterates its way to a solution. Play it again and the random start is reshuffled, a reminder that the final groups can depend on where you began.
Everything here is identical to the panel above, save for a single change: k is now three instead of five. Because k is something we choose rather than something the algorithm discovers, the choice is entirely ours, and it completely reshapes the outcome. With fewer flags to go around, the same penguins are swept into broader, coarser groups. There is no universally correct value — set k too small and genuinely distinct groups are forced to share a colour, set it too large and a single natural group is split in two. This echoes the balancing act we met with k in kNN, where too few neighbours overfit to noise and too many wash out the detail. In both algorithms the humble letter k is the dial that trades sharpness against stability.
Nothing obliges us to cluster on the same two measurements. Here we keep culmen length along the bottom but swap the vertical axis for flipper length, and the groups rearrange themselves accordingly — a reminder that clustering only ever knows about the features you feed it. We have also trimmed the view to flipper lengths between 170 and 235 mm and, crucially, discarded any penguin lying beyond that band altogether. Because each centroid is an average, a single wildly misplaced point can drag an entire cluster toward it.
So far every plot has shown just two measurements at a time, but nothing stops us clustering on more. Here is the same unclustered grey cloud of penguins from where we began, now spread across all three features the page has used: culmen length, culmen depth and flipper length together, in a cube you can spin by dragging it with the cursor.
Now we set k-means loose on that same cube. This is exactly the plot from earlier — Lloyd's algorithm with k set to three — only now the centroids wander through three dimensions instead of two, and the nearest-centre test is measured through space rather than across a flat plane. Press play and watch the three crosses drift from their random starting points to the heart of each group, the colours settling as they go; play again for a fresh random start. Note that we don't always identify the same three clusters accurately. Another point to consider is that while we can't display more than three dimensions easily here, one can cluster on many more than just three dimensions algorithmically.
Thank you to the providers of the data set, which you can find here on Kaggle. The Claude Opus LLM helped create this page. There are more algorithms and data structures to be found on the main page.
Please consider sharing this site on social media!