Bayesian Optimization with Optuna
optuna.create_study(), suggest_float/int/categorical, pruning, visualization.
Bayesian Optimization with Optuna is a free Learn AI with Python lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Smarter Than Random Search
Grid and random search ignore past results. Bayesian optimization learns from previous trials to propose more promising hyperparameters next, finding good values in fewer evaluations.
What Optuna Does
Optuna is a popular framework for hyperparameter optimization. You define an objective function; Optuna intelligently samples hyperparameters and remembers what worked.
Creating a Study
A study manages the optimization. Set direction to "maximize" for accuracy/AUC or "minimize" for loss/error.
import optuna
study = optuna.create_study(direction="maximize")The Objective Function
The objective receives a trial object, suggests hyperparameters, trains a model, and returns the score to optimize.
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score
def objective(trial):
n = trial.suggest_int("n_estimators", 100, 500)
d = trial.suggest_int("max_depth", 3, 20)
clf = RandomForestClassifier(n_estimators=n, max_depth=d)
return cross_val_score(clf, X, y, cv=3).mean()Suggesting Integer Parameters
trial.suggest_int(name, low, high) picks an integer in a range. Add step or log=True to control sampling.
def objective(trial):
n_estimators = trial.suggest_int("n_estimators", 100, 1000, step=50)
return train_and_score(n_estimators)Suggesting Float Parameters
trial.suggest_float(name, low, high) picks a real number. Use log=True for parameters like learning rate that span orders of magnitude.
def objective(trial):
lr = trial.suggest_float("learning_rate", 1e-4, 0.3, log=True)
subsample = trial.suggest_float("subsample", 0.5, 1.0)
return train_and_score(lr, subsample)Suggesting Categorical Parameters
trial.suggest_categorical(name, choices) selects from a fixed list, perfect for options like kernel type or boosting method.
def objective(trial):
kernel = trial.suggest_categorical("kernel", ["linear", "rbf", "poly"])
return train_svm(kernel)Running the Optimization
study.optimize(objective, n_trials=N) runs the search for N trials. More trials explore more of the space; Optuna focuses on promising regions over time.
study.optimize(objective, n_trials=100)Reading the Best Result
After optimizing, study.best_params gives the best hyperparameters and study.best_value the best objective score.
print("Best params:", study.best_params)
print("Best value:", study.best_value)
best = RandomForestClassifier(**study.best_params)
best.fit(X, y)Pruning Unpromising Trials
Optuna can stop bad trials early with pruners, saving compute. Report intermediate values and check trial.should_prune() inside training loops.
import optuna
study = optuna.create_study(
direction="maximize",
pruner=optuna.pruners.MedianPruner(),
)Why Optuna Scales
Optuna supports parallel trials, persistent storage, and visualization of the search. For expensive models, its sample-efficient Bayesian approach finds strong configurations with far fewer trials than grid or random search.
Quick Check
Test your Optuna knowledge.
Recap
Recap: Optuna uses Bayesian optimization to tune hyperparameters efficiently. Create a study with a direction, write an objective that uses trial.suggest_int/float/categorical, run study.optimize(objective, n_trials=N), then read study.best_params and study.best_value. Pruners stop weak trials to save compute.
Frequently asked questions
Is the “Bayesian Optimization with Optuna” lesson free?
Yes — the full text of “Bayesian Optimization with Optuna” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.
What will I learn in “Bayesian Optimization with Optuna”?
optuna.create_study(), suggest_float/int/categorical, pruning, visualization. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Learn AI with Python?
No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Bayesian Optimization with Optuna” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Learn AI with Python lesson?
Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Cross-Validation Strategies
- Classification Metrics Deep Dive
- Grid Search and Random Search
- Bayesian Optimization with Optuna