Load the Model Once at Startup
Use lifespan events so loading happens just once.
Load the Model Once at Startup is a free MLOps Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the MLOps Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Slow-Endpoint Trap
If you call joblib.load inside /predict, the model reloads on every request. That is slow and wasteful for large models. 🐢
Load It Just Once
The fix: load the model a single time when the service starts, keep it in memory, and reuse it for every prediction.
The Lifespan Hook
FastAPI gives you a lifespan function to run setup before serving and cleanup after. It is the right home for model loading.
Write an Async Context Manager
You decorate an async function with asynccontextmanager. Code before yield runs at startup; code after runs at shutdown.
from contextlib import asynccontextmanagerStash the Model in State
Load the model at startup and save it on app.state, a shared place every request handler can reach later.
@asynccontextmanager
async def lifespan(app):
app.state.model = joblib.load("model.joblib")
yieldWire It to the App
Pass your lifespan function when you create the app. Now FastAPI runs your loading code once as the server boots.
app = FastAPI(lifespan=lifespan)Read It in the Route
Inside /predict you grab the already-loaded model from app.state. No disk read, just a fast in-memory lookup.
@app.post("/predict")
def predict(req: Request):
model = req.app.state.modelClean Up at Shutdown
Anything after the yield runs when the app stops, perfect for closing files or freeing GPU memory the model held.
yield
app.state.model = NoneFaster First Request
Because loading finished at startup, even the very first user gets a fast response, not a cold-load penalty. ⚡
Watch the Worker Count
Each uvicorn worker is its own process with its own copy of the model. More workers means more memory, so size them deliberately.
Why It Matters
Loading once is a core serving optimization: it cuts latency, lowers disk I/O, and keeps memory predictable under real traffic. 📈
Quick Check
You want your model loaded exactly once when the service boots. Where should that happen?
Recap
You moved loading into a lifespan hook, stored the model on app.state, and read it per request. One load, fast responses, clean shutdown. 🙌
Frequently asked questions
Is the “Load the Model Once at Startup” lesson free?
Yes — the full text of “Load the Model Once at Startup” is free to read here on the web, and the MLOps Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the MLOps Academy course, upgrade to CoddyKit PRO.
What will I learn in “Load the Model Once at Startup”?
Use lifespan events so loading happens just once. You practise MLOps Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start MLOps Academy?
No prior experience is required. MLOps Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Load the Model Once at Startup” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this MLOps Academy lesson?
Yes. Every MLOps Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Your First /predict Endpoint
- Validate Requests with Pydantic
- Load the Model Once at Startup
- Add a /health Readiness Check