TorchScript와 torch.compile

Your training code is flexible Python, but servers want something fast and stable. TorchScript freezes your model into a portable, optimized form. 🚀

TorchScript is a serializable, optimizable subset of Python that runs without a Python interpreter, so your model can ship to C++ servers and mobile.

Tracing runs your model on an example input and records every operation it performs, capturing the path the data actually took.

Because tracing only follows one run, it silently ignores if-statements and loops that depend on the data. Those branches just vanish.

Scripting compiles your code directly, preserving every loop and condition. Use it when control flow depends on the input values.

A scripted or traced model saves to a single self-contained file you can load on any machine, no original class definition required.

The modern alternative is torch.compile, which speeds up your model with one line while keeping your code plain Python.

Under the hood torch.compile fuses many small operations into fewer optimized kernels, cutting overhead and boosting throughput on GPU.

The first forward pass after torch.compile is slow because it compiles in the background. Every call after that runs much faster. ⏳

Before you trace, script, or compile for serving, call model.eval() so dropout and batch norm behave correctly for inference.

Choose TorchScript when you need a Python-free deployment file, and torch.compile when you want a quick speedup inside Python.

You learned to freeze a model with TorchScript via tracing or scripting, save it anywhere, and accelerate inference in Python with torch.compile. 🎉