0Pricing
Machine Learning Academy · Lección

Cómo elegir k: método del codo y curvas de validación

Pruebe valores de k del 1 al 30, represente la precisión de validación e identifique el punto óptimo que equilibra el sesgo y la varianza.

Cómo elegir k: método del codo y curvas de validación es una lección gratuita de Machine Learning Academy en CoddyKit. Esta es la lección 2 de 4. Puedes leer la lección completa abajo gratuitamente — luego la practicas en el navegador con un editor de código integrado y un tutor de IA 24/7. Forma parte de la ruta de aprendizaje de Machine Learning Academy, y tu progreso se sincroniza en la web y la app de CoddyKit. El curso de Machine Learning Academy incluye 4 lecciones en total.

Partes de esta lección aún no han sido traducidas y se muestran en inglés.

Why Choosing k Matters

The value of k is the most important hyperparameter in KNN. Too small a k (e.g., k=1) makes the model highly sensitive to noise: each training point forms its own prediction region, causing the model to memorise noise rather than learn patterns. Too large a k over-smooths the decision boundary and may merge truly distinct classes. Finding the right k is a bias-variance problem: small k = low bias, high variance; large k = high bias, low variance. The elbow method and validation curves help identify the optimal k empirically.

# k=1: memorises training set perfectly
# Training accuracy = 100%, test accuracy low (overfitting)

# k=N (all neighbors): always predicts majority class
# Training accuracy = majority fraction (underfitting)

# Optimal k: somewhere in between
# Maximises test/validation accuracy

from sklearn.neighbors import KNeighborsClassifier
print('k=1  overfits (memorises noise)')
print('k=N  underfits (ignores all variation)')
print('Best k: maximises cross-validated accuracy')

Sweeping k Values: The Basic Loop

The simplest approach is to train KNN for a range of k values, evaluate each on a validation set, and pick the k with the highest validation accuracy. Scikit-learn makes this straightforward: loop over k from 1 to some maximum, fit and score each model. Always evaluate on a held-out validation set or use cross-validation — evaluating on the training set would always select k=1 (since with k=1 KNN predicts training points with 100% accuracy by memorising them).

from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_tr = scaler.fit_transform(X_train)
X_v  = scaler.transform(X_val)

val_scores = []
for k in range(1, 31):
    knn = KNeighborsClassifier(n_neighbors=k)
    knn.fit(X_tr, y_train)
    val_scores.append(knn.score(X_v, y_val))

best_k = val_scores.index(max(val_scores)) + 1
print('Best k:', best_k, 'with accuracy:', max(val_scores).round(3))

Plotting the Validation Curve

Visualising validation accuracy vs k reveals two important patterns. At low k, the curve is noisy with high variance (the model reacts to individual training points). As k increases, accuracy generally improves until a peak, then slowly degrades as the model becomes too smooth. The optimal k is at the peak of the validation curve. This shape is not always obvious with a single validation split, which is why cross-validation gives a more reliable estimate by averaging over multiple splits.

import matplotlib.pyplot as plt
import numpy as np

k_values = range(1, 31)
# val_scores computed from previous sweep
plt.figure(figsize=(10, 5))
plt.plot(k_values, val_scores, marker='o', label='Validation accuracy')
plt.axvline(x=best_k, color='r', linestyle='--', label=f'Best k={best_k}')
plt.xlabel('k (Number of Neighbors)')
plt.ylabel('Validation Accuracy')
plt.title('KNN Validation Curve')
plt.legend()
plt.grid(True)
plt.show()

Cross-Validated Accuracy per k

A single validation split may be biased by which samples happened to land in the validation set. Cross-validation averages across k splits, giving a more stable estimate. Scikit-learn's cross_val_score can be called for each k value. The mean accuracy shows the trend, and the standard deviation across folds shows reliability. Pick the k with the highest mean cross-validated accuracy; if multiple k values are close, prefer the larger one for smoother, more generalisable predictions.

from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
import numpy as np

X, y = load_iris(return_X_y=True)

cv_means, cv_stds = [], []
for k in range(1, 31):
    pipe = Pipeline([('sc', StandardScaler()),
                     ('knn', KNeighborsClassifier(n_neighbors=k))])
    scores = cross_val_score(pipe, X, y, cv=10)
    cv_means.append(scores.mean())
    cv_stds.append(scores.std())

best_k_cv = np.argmax(cv_means) + 1
print('Best k by CV:', best_k_cv, 'mean accuracy:', max(cv_means).round(3))

The Elbow Method Concept

The elbow method is a visual technique for finding the sweet spot where accuracy improvement becomes marginal. Plot validation accuracy (or error) vs k; the curve typically shows steep improvement at small k, then levels off. The elbow — the point where the curve bends from steep to flat — is often the optimal k. The intuition is diminishing returns: adding more neighbors beyond this point does not improve accuracy meaningfully but does increase bias. The elbow is not always obvious, which is why quantitative methods like CV are preferred for final selection.

import matplotlib.pyplot as plt
import numpy as np

k_range = range(1, 31)
error_rates = [1 - acc for acc in cv_means]  # Convert accuracy to error

plt.figure(figsize=(10, 5))
plt.plot(list(k_range), error_rates, marker='o')
plt.xlabel('k')
plt.ylabel('Cross-Validated Error Rate')
plt.title('Elbow Method for Optimal k')
plt.grid(True)

# Mark the elbow visually
plt.axvline(x=best_k_cv, color='r', linestyle='--', label=f'Elbow at k={best_k_cv}')
plt.legend()
plt.show()

Using validation_curve from scikit-learn

Scikit-learn provides validation_curve() as a convenience function that sweeps a hyperparameter range and returns training and validation scores for each value. It is cleaner than a manual loop because it handles cross-validation internally. The param_name argument uses double-underscore notation for pipeline parameters (e.g., knn__n_neighbors). Plotting both train and validation curves together reveals whether low performance is due to underfitting (both low), overfitting (train high, val low), or a good generalisation (both high).

from sklearn.model_selection import validation_curve
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
import numpy as np

pipe = Pipeline([('sc', StandardScaler()), ('knn', KNeighborsClassifier())])

train_scores, val_scores = validation_curve(
    pipe, X, y,
    param_name='knn__n_neighbors',
    param_range=range(1, 31),
    cv=10, scoring='accuracy'
)

train_mean = np.mean(train_scores, axis=1)
val_mean   = np.mean(val_scores, axis=1)

print('Best k:', np.argmax(val_mean) + 1)

Bias-Variance Trade-Off in KNN

The validation curve illustrates the bias-variance trade-off directly. For small k: training accuracy approaches 100% (low bias, model fits training data perfectly) while validation accuracy is lower (high variance, model is too sensitive to individual points). For large k: training accuracy drops (model underfits) and validation accuracy also drops (high bias). The optimal k sits at the intersection where the gap between training and validation accuracy is small and both are maximised — this is the generalisation sweet spot.

import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots(figsize=(10, 5))
k_range = list(range(1, 31))

ax.plot(k_range, train_mean, label='Training accuracy', color='blue')
ax.plot(k_range, val_mean, label='Validation accuracy', color='orange')
ax.fill_between(k_range,
    np.mean(train_scores, axis=1) - np.std(train_scores, axis=1),
    np.mean(train_scores, axis=1) + np.std(train_scores, axis=1),
    alpha=0.1, color='blue')

ax.set_xlabel('k')
ax.set_ylabel('Accuracy')
ax.set_title('Bias-Variance Trade-off: KNN Validation Curve')
ax.legend()
plt.show()

Choosing k Based on Odd vs Even

For binary classification, always prefer odd values of k to avoid ties. With k=4 and two classes, you might get 2 votes each — the tie-breaking rule then determines the result, which can be arbitrary. By using k=3 or k=5, ties are impossible with two classes. For multi-class problems with C classes, k should not be a multiple of C for the same reason. This is a small but practically important detail when k values near the optimum are close in performance.

# Best practice for binary classification: pick odd k
# For multi-class (C classes): avoid multiples of C

def recommend_k(k_optimal, n_classes):
    if n_classes == 2:
        # Make odd
        return k_optimal if k_optimal % 2 == 1 else k_optimal + 1
    else:
        # Avoid multiples of n_classes
        while k_optimal % n_classes == 0:
            k_optimal += 1
        return k_optimal

print('Binary, k=4 -> recommended:', recommend_k(4, 2))  # 5
print('3-class, k=6 -> recommended:', recommend_k(6, 3)) # 7

GridSearchCV for k Selection

GridSearchCV automates k selection by evaluating every candidate value through cross-validation and returning the best one. Combine it with a scaling pipeline and pass the param_grid with the double-underscore key. GridSearchCV also retrains the best model on the full training set, so grid.best_estimator_ is ready for production use immediately after fitting. This is the recommended approach when k selection is part of a larger hyperparameter optimisation.

from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier

pipe = Pipeline([
    ('sc', StandardScaler()),
    ('knn', KNeighborsClassifier())
])

param_grid = {
    'knn__n_neighbors': list(range(1, 31, 2)),  # odd values 1-29
    'knn__weights': ['uniform', 'distance']
}

grid = GridSearchCV(pipe, param_grid, cv=10, scoring='accuracy', n_jobs=-1)
grid.fit(X_train, y_train)

print('Best k:', grid.best_params_['knn__n_neighbors'])
print('Best weights:', grid.best_params_['knn__weights'])
print('Best CV accuracy:', grid.best_score_.round(3))

Interpreting Results and Making Final k Choice

When multiple k values produce similar validation scores, prefer the larger k for smoother, more robust predictions that are less sensitive to individual noise points. Inspect the standard deviation of cross-validation scores: if a smaller k has a higher mean but also higher std, the larger k may actually be more reliable in production. The final model should be retrained on the entire training set (not just the cross-validation training folds) using the selected k and evaluated once on the held-out test set.

import numpy as np
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

# Select best k from validation
best_k = 11  # determined from CV

# Retrain on full training data
final_model = Pipeline([
    ('sc', StandardScaler()),
    ('knn', KNeighborsClassifier(n_neighbors=best_k, weights='distance'))
])
final_model.fit(X_train, y_train)

# Evaluate once on held-out test set
test_accuracy = final_model.score(X_test, y_test)
print(f'Final test accuracy with k={best_k}: {test_accuracy:.3f}')

Common Pitfalls When Choosing k

Three common mistakes to avoid: (1) Evaluating on training data — k=1 will always score 100%, making it appear optimal; always use held-out or cross-validated data. (2) Not scaling features before choosing k — the optimal k depends on the distance geometry, which changes with scaling; always include the scaler in the pipeline before the hyperparameter search. (3) Choosing k independently of the dataset size — a rule of thumb is to start with k around sqrt(N) where N is training set size, then refine through CV.

import numpy as np

N_train = 1000  # training samples

# Rule of thumb starting point
k_start = int(np.sqrt(N_train))
print(f'sqrt(N) starting point: k = {k_start}')

# Then sweep around this value
k_candidates = list(range(max(1, k_start - 10), k_start + 11, 2))
print('Candidates to sweep:', k_candidates)

# AVOID:
# knn.score(X_train, y_train) -- always pick k=1
# Not including scaler in pipeline before CV

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: how small k causes overfitting and large k causes underfitting, how to use cross-validation and validation_curve to sweep k and find the optimal value, and why odd k values prevent ties in binary classification. Next up we explore different distance metrics — Euclidean, Manhattan, and Minkowski — and how to choose between them.

Preguntas frecuentes

¿La lección «Cómo elegir k: método del codo y curvas de validación» es gratis?

Sí — el texto completo de «Cómo elegir k: método del codo y curvas de validación» es gratis para leer aquí en la web. Para practicarla de forma interactiva (editor de código integrado y tutor de IA 24/7) y desbloquear el resto del curso de Machine Learning Academy, actualiza a CoddyKit PRO. El curso de Machine Learning Academy incluye 4 lecciones en total.

¿Qué aprenderé en «Cómo elegir k: método del codo y curvas de validación»?

Pruebe valores de k del 1 al 30, represente la precisión de validación e identifique el punto óptimo que equilibra el sesgo y la varianza. Practicas Machine Learning Academy con código real que ejecutas directamente en el navegador, y un tutor de IA 24/7 responde tus preguntas mientras trabajas en la lección.

¿Necesito experiencia previa para empezar Machine Learning Academy?

No se requiere experiencia previa. Machine Learning Academy en CoddyKit está estructurado para principiantes hasta estudiantes avanzados, así que puedes empezar aquí o desde el inicio y avanzar a tu ritmo. Esta es la lección 2 de 4.

¿Cuánto tiempo toma la lección «Cómo elegir k: método del codo y curvas de validación»?

La mayoría de las lecciones de CoddyKit toman alrededor de 5–10 minutos. Cada una es compacta e interactiva, así que avanzas constantemente y retomas exactamente por donde dejaste en la web y la app.

¿Puedo escribir y ejecutar código en esta lección de Machine Learning Academy?

Sí. Cada lección de Machine Learning Academy incluye un editor de código integrado, así que escribes y ejecutas código real directamente en tu navegador y obtienes retroalimentación instantánea de IA — sin configuración local necesaria.

Todas las lecciones de este curso

  1. Cómo funciona KNN: distancias, vecinos y votos
  2. Cómo elegir k: método del codo y curvas de validación
  3. Métricas de distancia: euclídea, Manhattan y Minkowski
  4. KNN para regresión y sus límites de escalabilidad
← Volver a Machine Learning Academy