التقاط العمل في رسم بياني

A CUDA graph records a whole sequence of kernels and copies as one reusable unit of work you can submit again and again.

Each individual kernel launch has a tiny CPU overhead. Across thousands of launches per iteration, that cost becomes real.

A graph lets you describe the work once, then replay it cheaply, paying the setup cost a single time.

The easiest way to build one is stream capture: you record the normal launches you already issue into a stream.

Between begin and end, you launch kernels and copies as usual. Nothing runs yet; the operations are recorded, not executed.

You stop recording with cudaStreamEndCapture, which hands back a cudaGraph_t describing all the work.

Capture figures out the dependencies between operations automatically from how you used the stream.

A graph is just a description. You compile it into a runnable graph exec object before you can launch it.

You can also add nodes explicitly with the graph API for full control over structure and edges.

Capture can span several streams, so independent branches that run concurrently all land in the same graph.

Once instantiated, the same exec object is launched repeatedly, which is exactly where graphs earn their keep.

What does cudaStreamEndCapture produce?

Wrap launches in stream capture to record a cudaGraph_t, instantiate it once, and capture infers the dependencies for you. Great progress! 🎉