Cara Kerja KNN: Jarak, Tetangga, dan Suara
Peserta didik akan memvisualisasikan dataset 2D, menghitung jarak Euclidean, mengidentifikasi k tetangga terdekat, dan menghasilkan klasifikasi berdasarkan suara mayoritas.
Cara Kerja KNN: Jarak, Tetangga, dan Suara adalah pelajaran Machine Learning Academy gratis di CoddyKit. Ini adalah pelajaran 1 dari 4. Kamu bisa membaca pelajaran lengkapnya di bawah secara gratis — lalu praktikkan langsung di browser dengan editor kode bawaan dan tutor AI 24/7. Ini adalah bagian dari jalur belajar Machine Learning Academy, dan progresmu tersinkronisasi di web dan aplikasi CoddyKit. Kursus Machine Learning Academy mencakup 4 pelajaran total.
Bagian dari pelajaran ini belum diterjemahkan dan ditampilkan dalam bahasa Inggris.
The Core Intuition Behind KNN
K-Nearest Neighbors (KNN) is one of the most intuitive machine learning algorithms: predict the label of a new point by looking at the k closest labelled points and taking a majority vote. There is no explicit training phase — the algorithm simply memorises the training data and performs all computation at prediction time. This makes KNN a lazy learner. It works well when similar inputs have similar outputs, which is a reasonable assumption in many real-world problems like recommending products or diagnosing diseases.
# Conceptual pseudocode
def knn_predict(X_train, y_train, x_new, k=3):
# 1. Compute distance from x_new to every training point
distances = [euclidean(x_new, x_i) for x_i in X_train]
# 2. Find indices of k smallest distances
nearest = sorted(range(len(distances)), key=lambda i: distances[i])[:k]
# 3. Majority vote among k neighbors
votes = [y_train[i] for i in nearest]
return max(set(votes), key=votes.count)Euclidean Distance: The Default Metric
The most common distance measure in KNN is Euclidean distance, which is the straight-line distance between two points in feature space. For two points A=(a1, a2) and B=(b1, b2), the Euclidean distance is sqrt((a1-b1)^2 + (a2-b2)^2). In higher dimensions, the same formula extends across all features. Because Euclidean distance treats all dimensions equally, features must be on the same scale — otherwise high-magnitude features dominate the distance calculation and KNN performs poorly.
import numpy as np
def euclidean_distance(a, b):
return np.sqrt(np.sum((a - b) ** 2))
point_a = np.array([1.0, 2.0])
point_b = np.array([4.0, 6.0])
dist = euclidean_distance(point_a, point_b)
print('Euclidean distance:', dist) # 5.0
# Verify with numpy
print('Using numpy:', np.linalg.norm(point_a - point_b))Finding the k Nearest Neighbors
Given a query point, KNN computes distances to all N training points, sorts them, and selects the top k closest. On a small 2D dataset you can visualise this by drawing a circle around the new point that expands until it encloses exactly k training samples — those are the neighbors. The computational cost is O(N * d) per prediction, where N is the number of training points and d is the number of features. This is fine for small datasets but becomes prohibitively slow on large ones.
import numpy as np
# Training data
X_train = np.array([[1,2],[2,3],[3,1],[6,5],[7,7],[8,6]])
y_train = np.array([0, 0, 0, 1, 1, 1]) # 0=class A, 1=class B
# Query point
x_new = np.array([4, 4])
# Distances to all training points
dists = np.linalg.norm(X_train - x_new, axis=1)
print('Distances:', dists.round(2))
# Indices of 3 nearest
k = 3
nearest_idx = np.argsort(dists)[:k]
print('3 nearest labels:', y_train[nearest_idx])Classification by Majority Vote
After finding the k nearest neighbors, KNN for classification assigns the class with the most votes among the neighbors. For k=3, if 2 neighbors are class A and 1 is class B, the prediction is class A. Ties are broken by the implementation (usually by choosing the class with the closest single neighbor). For regression, KNN averages the target values of the k neighbors instead of voting. Choosing k=1 is most flexible but very noisy; larger k is smoother but may underfit.
from collections import Counter
import numpy as np
neighbor_labels = np.array([0, 0, 1]) # 2 votes for class 0, 1 for class 1
# Majority vote
vote_counts = Counter(neighbor_labels)
prediction = vote_counts.most_common(1)[0][0]
print('Predicted class:', prediction) # 0
# For regression: average instead of vote
neighbor_values = np.array([15.2, 18.7, 14.1])
prediction_reg = np.mean(neighbor_values)
print('Regression prediction:', prediction_reg.round(2))KNN with scikit-learn: KNeighborsClassifier
Scikit-learn implements KNN through KNeighborsClassifier and KNeighborsRegressor. These classes follow the standard fit/predict API. Key parameters include n_neighbors (k value), metric (distance function), and weights (uniform or distance-weighted voting). Distance-weighted voting (weights='distance') gives closer neighbors more influence, which often improves performance by reducing the impact of the farthest (least similar) neighbors in the chosen k.
from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)
knn = KNeighborsClassifier(n_neighbors=5, weights='uniform')
knn.fit(X_train_s, y_train)
print('Test accuracy:', knn.score(X_test_s, y_test).round(3))Visualising the Decision Boundary
KNN's decision boundary is inherently non-linear and local. With k=1, the boundary follows training data exactly (creating jagged Voronoi-like regions), while larger k produces smoother boundaries. You can visualise this on a 2D dataset by predicting over a fine mesh grid and colouring each region by predicted class. Low k overfits the training noise (every point is its own class island); high k smooths too aggressively, potentially merging distinct clusters. The ideal k balances this bias-variance trade-off.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.neighbors import KNeighborsClassifier
# Create 2D mesh for decision boundary
def plot_decision_boundary(clf, X, y):
h = 0.02
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(
np.arange(x_min, x_max, h),
np.arange(y_min, y_max, h)
)
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, alpha=0.4)
plt.scatter(X[:, 0], X[:, 1], c=y)Why Feature Scaling Is Critical for KNN
KNN computes distances in feature space, so the scale of each feature directly affects which points are considered nearest. If one feature has values in the thousands (e.g., income) and another in single digits (e.g., number of children), the high-magnitude feature will dominate all distance calculations. A neighbour that differs by 1 in income but is identical in all other features might be considered farther away than one that differs drastically in all other features. Always apply StandardScaler or MinMaxScaler before KNN.
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.model_selection import cross_val_score
from sklearn.datasets import load_breast_cancer
X, y = load_breast_cancer(return_X_y=True)
# Without scaling
knn_raw = KNeighborsClassifier(n_neighbors=5)
raw_score = cross_val_score(knn_raw, X, y, cv=5).mean()
# With scaling inside pipeline
pipe = Pipeline([('sc', StandardScaler()), ('knn', KNeighborsClassifier(n_neighbors=5))])
scaled_score = cross_val_score(pipe, X, y, cv=5).mean()
print(f'Without scaling: {raw_score:.3f}')
print(f'With scaling: {scaled_score:.3f}')Getting Neighbor Distances and Indices
Sometimes you need more than just the predicted class — you need to know which training examples were the neighbors and how far away they were. KNN's kneighbors() method returns both the distances and the indices of the nearest training samples. This is useful for anomaly detection (large average distance to neighbors suggests an outlier), recommendation systems, and explaining predictions to end users by showing the most similar known examples.
from sklearn.neighbors import KNeighborsClassifier
import numpy as np
X_train = np.array([[1,2],[2,3],[5,5],[8,7]])
y_train = np.array([0, 0, 1, 1])
knn = KNeighborsClassifier(n_neighbors=2)
knn.fit(X_train, y_train)
x_query = np.array([[4, 4]])
distances, indices = knn.kneighbors(x_query)
print('Neighbor indices:', indices)
print('Distances to neighbors:', distances.round(2))
print('Neighbor labels:', y_train[indices[0]])Weighted Voting by Distance
Uniform voting treats all k neighbors equally regardless of how close they are. Distance-weighted voting (weights='distance') gives each neighbor a weight proportional to the inverse of its distance — very close neighbors have much more influence than far ones. This is particularly useful at decision boundaries where the nearest neighbor and the farthest neighbor may be on different sides of the true boundary. Weighted KNN almost always outperforms uniform KNN, especially when k is large.
from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_digits
from sklearn.model_selection import cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
X, y = load_digits(return_X_y=True)
uniform_pipe = Pipeline([('sc', StandardScaler()),
('knn', KNeighborsClassifier(n_neighbors=7, weights='uniform'))])
distance_pipe = Pipeline([('sc', StandardScaler()),
('knn', KNeighborsClassifier(n_neighbors=7, weights='distance'))])
print('Uniform: ', cross_val_score(uniform_pipe, X, y, cv=5).mean().round(3))
print('Distance: ', cross_val_score(distance_pipe, X, y, cv=5).mean().round(3))Predicting Class Probabilities
Instead of a hard class label, KNN can return class probabilities using predict_proba(). For k=5, if 3 neighbors are class 1 and 2 are class 0, the predicted probability is [0.4, 0.6]. These probabilities can be thresholded for precision-recall control, used in ensemble models, or calibrated with CalibratedClassifierCV if the raw fractions do not represent true probabilities. For distance-weighted KNN, probabilities are weighted sums rather than simple fractions.
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
import numpy as np
X_train = np.array([[1,1],[1,2],[5,5],[6,5],[5,6]])
y_train = np.array([0, 0, 1, 1, 1])
scaler = StandardScaler()
X_s = scaler.fit_transform(X_train)
knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(X_s, y_train)
x_new = scaler.transform([[3, 3]])
proba = knn.predict_proba(x_new)
print('Class probabilities:', proba)
print('Predicted class:', knn.predict(x_new))KNN Strengths and Weaknesses
KNN has several strengths: it is easy to understand, requires no training time, naturally handles multi-class problems, and can model complex non-linear boundaries. Its weaknesses are significant for large datasets: prediction is slow at O(N * d) per query, it requires all training data to be in memory, and it degrades in high dimensions (the curse of dimensionality). KNN is a strong baseline for small to medium datasets with well-scaled features, but is typically replaced by faster models in production at scale.
# Summary of KNN trade-offs
strengths = [
'No training time -- all computation at prediction',
'No assumptions about data distribution',
'Naturally handles multi-class classification',
'Non-linear decision boundary',
]
weaknesses = [
'Slow prediction: O(N*d) per query',
'High memory: stores all training data',
'Sensitive to irrelevant and scaled features',
'Poor in very high dimensions (curse of dimensionality)',
]
for s in strengths: print('+', s)
for w in weaknesses: print('-', w)Quick Check
Test your understanding of Machine Learning with Python concepts from this lesson.
Lesson Recap
In this lesson you learned: how KNN classifies by majority vote among k nearest neighbors using Euclidean distance, why feature scaling is critical before applying KNN, and how distance-weighted voting and predict_proba work for more nuanced predictions. Next up we explore how to choose the best k using the elbow method and validation curves.
Pertanyaan yang Sering Diajukan
Apakah pelajaran “Cara Kerja KNN: Jarak, Tetangga, dan Suara” gratis?
Ya — teks lengkap “Cara Kerja KNN: Jarak, Tetangga, dan Suara” gratis dibaca di sini di web. Untuk praktiknya secara interaktif (editor kode bawaan dan tutor AI 24/7) dan buka sisa kursus Machine Learning Academy, upgrade ke CoddyKit PRO. Kursus Machine Learning Academy mencakup 4 pelajaran total.
Apa yang akan aku pelajari di “Cara Kerja KNN: Jarak, Tetangga, dan Suara”?
Peserta didik akan memvisualisasikan dataset 2D, menghitung jarak Euclidean, mengidentifikasi k tetangga terdekat, dan menghasilkan klasifikasi berdasarkan suara mayoritas. Kamu berlatih Machine Learning Academy dengan kode praktik yang langsung kamu jalankan di browser, dan tutor AI 24/7 menjawab pertanyaanmu saat kamu mengerjakan pelajaran ini.
Apakah aku perlu pengalaman untuk memulai Machine Learning Academy?
Tidak diperlukan pengalaman sebelumnya. Machine Learning Academy di CoddyKit dirancang untuk pemula hingga pelajar tingkat lanjut, jadi kamu bisa memulai di sini atau dari awal dan belajar sesuai kecepatan kamu sendiri. Ini adalah pelajaran 1 dari 4.
Berapa lama pelajaran “Cara Kerja KNN: Jarak, Tetangga, dan Suara” memakan waktu?
Sebagian besar pelajaran CoddyKit memakan waktu sekitar 5–10 menit. Setiap pelajaran ringkas dan interaktif, jadi kamu membuat kemajuan stabil dan melanjutkan dari tempat kamu tinggalkan di web dan aplikasi.
Bisakah aku menulis dan menjalankan kode dalam pelajaran Machine Learning Academy ini?
Ya. Setiap pelajaran Machine Learning Academy menyertakan editor kode bawaan, jadi kamu menulis dan menjalankan kode nyata langsung di browser dan mendapatkan umpan balik AI instan — tidak diperlukan penyiapan lokal.
Semua pelajaran dalam kursus ini
- Cara Kerja KNN: Jarak, Tetangga, dan Suara
- Memilih k: Metode Siku dan Kurva Validasi
- Metrik Jarak: Euclidean, Manhattan, dan Minkowski
- KNN untuk Regresi dan Batas Skalabilitasnya