t-SNE: Neighbourhood Preservation for Visualisation
Learners will apply t-SNE with different perplexity settings to MNIST embeddings and understand that t-SNE distances are not meaningful for downstream modelling.
t-SNE: Neighbourhood Preservation for Visualisation is a free Machine Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Machine Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Beyond PCA: Non-Linear Visualisation
PCA projects data linearly and preserves global variance, but can fail to show local cluster structure. t-SNE (t-distributed Stochastic Neighbour Embedding) is a non-linear dimensionality reduction technique designed specifically for 2D and 3D visualisation. It prioritises preserving local neighbourhoods: points that are close in high-dimensional space should also be close in the 2D plot.
The Core Idea: Similarity Distributions
t-SNE defines a probability distribution over pairs of points in high-dimensional space: nearby points have high similarity. It then defines a similar distribution in the low-dimensional embedding. The algorithm minimises the KL divergence between the two distributions using gradient descent, nudging points in 2D until the neighbourhood structure matches the high-D structure.
The Perplexity Parameter
Perplexity is t-SNE's most important hyperparameter. It loosely controls how many neighbours each point considers when building the high-D similarity distribution — typically between 5 and 50. Low perplexity focuses on very local structure (many small clusters); high perplexity captures more global structure (broader, more spread-out clusters). The same dataset can look very different under different perplexity values.
Running t-SNE in scikit-learn
Use sklearn.manifold.TSNE. Key parameters: n_components (almost always 2), perplexity, n_iter (default 1000), and random_state. t-SNE is computationally expensive — O(n² log n) — so reduce the dataset with PCA first for large inputs (e.g., PCA to 50 dimensions, then t-SNE to 2D).
from sklearn.manifold import TSNE
from sklearn.datasets import load_digits
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
X, y = load_digits(return_X_y=True)
X_scaled = StandardScaler().fit_transform(X)
# Pre-reduce with PCA for speed
X_pca = PCA(n_components=30).fit_transform(X_scaled)
# t-SNE to 2D
tsne = TSNE(n_components=2, perplexity=30, n_iter=1000, random_state=42)
X_tsne = tsne.fit_transform(X_pca)
print('t-SNE shape:', X_tsne.shape)Visualising t-SNE Embeddings
Plot the 2D t-SNE coordinates with class colour to reveal cluster structure. On MNIST digits with appropriate perplexity, you typically see well-separated digit clusters, with similar-looking digits (e.g., 3 and 8) placed close together. This confirms that t-SNE captures semantically meaningful groupings.
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(8, 6))
scatter = ax.scatter(X_tsne[:, 0], X_tsne[:, 1], c=y, cmap='tab10', s=8, alpha=0.7)
fig.colorbar(scatter, ax=ax, label='Digit')
ax.set_title('MNIST digits — t-SNE (perplexity=30)')
ax.set_xlabel('t-SNE 1')
ax.set_ylabel('t-SNE 2')
plt.tight_layout()
plt.show()Effect of Perplexity on the Embedding
It is essential to try multiple perplexity values and compare the plots. A perplexity that is too low creates many small disconnected blobs even within the same true cluster. A perplexity that is too high smears clusters together. A good practice is to test perplexity in [5, 15, 30, 50] and choose the embedding where known cluster structure appears most clearly.
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE
perplexities = [5, 15, 30, 50]
fig, axes = plt.subplots(1, 4, figsize=(16, 4))
for ax, perp in zip(axes, perplexities):
tsne = TSNE(n_components=2, perplexity=perp, n_iter=800, random_state=0)
X_emb = tsne.fit_transform(X_pca[:300]) # subset for speed
ax.scatter(X_emb[:, 0], X_emb[:, 1], c=y[:300], cmap='tab10', s=10)
ax.set_title(f'Perplexity={perp}')
ax.axis('off')
plt.tight_layout()
plt.show()t-SNE Distances Are Not Meaningful
A critical warning: distances between clusters in t-SNE are not interpretable. A cluster appearing far from another does not mean they are globally distant; the algorithm optimises local neighbourhood preservation, not global distances. You cannot compare cluster sizes or inter-cluster distances across different runs or perplexity settings. Use t-SNE for exploration only, not for quantitative analysis.
t-SNE Is Stochastic and Non-Deterministic
Every t-SNE run with a different random_state produces a different layout — the embedding can rotate, reflect, or rearrange clusters. Always set random_state for reproducibility. Also, t-SNE does not have a transform method for out-of-sample points: you must refit on the entire dataset each time, which makes it unsuitable as a preprocessing step for a production model.
UMAP: A Modern Alternative to t-SNE
UMAP (Uniform Manifold Approximation and Projection) is a newer technique that is faster than t-SNE, preserves both local and more global structure, and supports transform for new points. It is not in scikit-learn but is installed via pip install umap-learn. For large datasets or production pipelines, UMAP is generally preferred over t-SNE.
# pip install umap-learn
import umap
reducer = umap.UMAP(n_components=2, n_neighbors=15, min_dist=0.1, random_state=42)
X_umap = reducer.fit_transform(X_pca)
import matplotlib.pyplot as plt
plt.scatter(X_umap[:, 0], X_umap[:, 1], c=y, cmap='tab10', s=8)
plt.title('MNIST — UMAP embedding')
plt.colorbar(label='Digit')
plt.show()When to Use t-SNE vs PCA
Use PCA for: preprocessing before modelling, compression, anomaly detection via reconstruction error, or when you need a deterministic, reversible transform. Use t-SNE for: exploring cluster structure in high-dimensional data, generating visualisations for presentations, or confirming that a dataset has meaningful groupings before applying a clustering or classification algorithm.
Complete t-SNE Visualisation Pipeline
Here is the recommended pipeline for t-SNE on any high-dimensional dataset: scale, reduce with PCA to ~50 dimensions, then apply t-SNE to 2D, and plot with class labels.
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
from sklearn.datasets import load_digits
import matplotlib.pyplot as plt
X, y = load_digits(return_X_y=True)
# Step 1: scale
X_s = StandardScaler().fit_transform(X)
# Step 2: PCA pre-reduction
X_pca = PCA(n_components=30, random_state=0).fit_transform(X_s)
# Step 3: t-SNE
X_tsne = TSNE(n_components=2, perplexity=30, random_state=0).fit_transform(X_pca)
# Step 4: plot
plt.scatter(X_tsne[:, 0], X_tsne[:, 1], c=y, cmap='tab10', s=10)
plt.title('Digits t-SNE')
plt.colorbar(label='Digit')
plt.show()Quick Check
Test your understanding of t-SNE from this lesson.
Lesson Recap
In this lesson you learned: t-SNE preserves local neighbourhoods by minimising KL divergence between high-D and low-D similarity distributions, perplexity controls the effective number of neighbours and should be tuned between 5 and 50, and t-SNE distances between clusters are not quantitatively meaningful — use it for exploration only. Next up we embed PCA inside a scikit-learn Pipeline as a preprocessing step for classifiers.
Frequently asked questions
Is the “t-SNE: Neighbourhood Preservation for Visualisation” lesson free?
Yes — the full text of “t-SNE: Neighbourhood Preservation for Visualisation” is free to read here on the web, and the Machine Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Machine Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “t-SNE: Neighbourhood Preservation for Visualisation”?
Learners will apply t-SNE with different perplexity settings to MNIST embeddings and understand that t-SNE distances are not meaningful for downstream modelling. You practise Machine Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Machine Learning Academy?
No prior experience is required. Machine Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “t-SNE: Neighbourhood Preservation for Visualisation” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Machine Learning Academy lesson?
Yes. Every Machine Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- PCA: Variance, Eigenvectors, and Principal Components
- Projecting Data and Reconstructing from Components
- t-SNE: Neighbourhood Preservation for Visualisation
- PCA as Preprocessing: Speed and Noise Reduction in Pipelines