데이터 누수를 시작 전에 막기

Data leakage is when test information sneaks into training. Scores look amazing, then collapse in the real world.

Leakage lets the model peek at answers it should not see. The test score becomes a fantasy, not a forecast of future performance.

Scaling or filling values on the whole dataset before splitting leaks. The test rows quietly shaped your preprocessing statistics.

Always fit transformers on the training data alone. Learn the mean, scale, or fill from train, never from the test set.

Once fitted on train, you simply transform the test set with those learned numbers. Test data is reshaped, never consulted.

A sneakier kind is target leakage: a feature that secretly encodes the answer, like using a refund flag to predict refunds.

In time data, using a value recorded after the prediction moment leaks the future. Only use information available at decision time.

Wrap preprocessing and the model in a Pipeline. It refits transforms on each training fold, blocking leakage automatically.

Pass the whole pipeline to cross-validation. Scaling is then learned inside each fold, so no test fold ever touches the fit.

A near-perfect score is a red flag, not a trophy. Real problems are noisy, so investigate before you celebrate.

Anything learned from data must come from training only. Keep the test set sealed until the final, single evaluation. 🔒

You scale your data. How do you avoid leakage?

Leakage inflates scores by leaking answers in. Fit transforms on train only, use a Pipeline, and keep the test set sealed. 🔒