K-Means Clustering

K-Means is one of the most popular clustering algorithms. It partitions data into a predefined number of clusters (K) by minimizing the intra-cluster distances.

It works iteratively to find the best cluster assignments and centroids.

K-Means operates in the following steps:

1. Initialize K random centroids.
2. Assign each data point to the nearest centroid.
3. Recalculate centroids as the mean of all points in a cluster.
4. Repeat steps 2 and 3 until centroids stabilize or a maximum number of iterations is reached.

The value of K is a critical parameter. Common methods to choose K include:

• Elbow Method: Plot the within-cluster sum of squares (WCSS) for different K values and look for the 'elbow point.'
• Silhouette Score: Measures how well-separated the clusters are.

Let’s consider a dataset with points distributed in 2D space. Each point has:

• X: Coordinate on the horizontal axis.
• Y: Coordinate on the vertical axis.

K-Means will group these points into clusters based on their proximity.

We’ll use the KMeans class from the scikit-learn library to implement the algorithm:

from sklearn.cluster import KMeans
import numpy as np

# Example dataset
X = np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]])

# K-Means model
kmeans = KMeans(n_clusters=2, random_state=0)
kmeans.fit(X)

print("Cluster Centers:", kmeans.cluster_centers_)
print("Labels:", kmeans.labels_)

We can visualize the clusters by plotting the data points and centroids:

import matplotlib.pyplot as plt

# Scatter plot of data points
plt.scatter(X[:, 0], X[:, 1], c=kmeans.labels_, cmap='viridis')

# Plot cluster centers
plt.scatter(kmeans.cluster_centers_[:, 0], kmeans.cluster_centers_[:, 1], s=200, c='red', marker='X')
plt.title('K-Means Clustering')
plt.show()

K-Means has several benefits:

• Simple and easy to implement.
• Efficient for large datasets.
• Works well when clusters are clearly defined.

Some limitations of K-Means include:

• Sensitivity to the initial placement of centroids.
• Difficulty in handling non-spherical clusters.
• Sensitivity to outliers and noisy data.

In this lesson, we covered:

• How K-Means clustering works.
• Choosing the number of clusters using methods like the elbow method.
• Implementing and visualizing K-Means in Python.

Next, we will apply K-Means to a real-world dataset to gain practical experience.