Profile and Tune Inference Latency

You cannot tune what you do not measure. Before changing settings, capture real latency and throughput numbers under load you trust. 📏

Triton ships with perf_analyzer, a tool that hammers your model with requests and reports latency and throughput so you can compare configs.