Learn AI with Python · Lesson

PyTorch Tensors and Autograd

torch.Tensor, requires_grad, backward(), gradient computation, tensor operations.

PyTorch Tensors and Autograd is a free Learn AI with Python lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What is PyTorch?

PyTorch is a deep learning framework built around tensors, multi-dimensional arrays with GPU acceleration and automatic differentiation. The two pillars you must master first are tensors and autograd.

pip install torch

import torch

Creating Tensors

A tensor is like a NumPy array but can live on a GPU and track gradients. Create one with torch.tensor. Tensors have a shape and a dtype.

x = torch.tensor([1.0, 2.0, 3.0])
print(x.shape)   # torch.Size([3])
print(x.dtype)   # torch.float32

Tensor Operations

Tensors support elementwise math, matrix multiplication, and reshaping. Operations are vectorized and run fast on hardware. This is the computational substrate of every neural network.

a = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
b = torch.tensor([[5.0, 6.0], [7.0, 8.0]])
print(a + b)
print(a @ b)  # matrix multiply

requires_grad: Tracking Gradients

To compute gradients, create a tensor with requires_grad=True. PyTorch then records every operation on it, building a computation graph it can differentiate later.

x = torch.tensor([2.0, 3.0], requires_grad=True)
print(x.requires_grad)  # True

Building a Computation

Apply operations to a grad-tracking tensor to produce a scalar output. Here we square each element and sum, giving a single number we can differentiate with respect to x.

x = torch.tensor([2.0, 3.0], requires_grad=True)
y = x.pow(2).sum()   # y = x0^2 + x1^2
print(y)             # tensor(13., grad_fn=...)

Calling backward()

Call .backward() on a scalar to compute gradients via backpropagation. PyTorch walks the recorded graph in reverse, filling in derivatives automatically.

y.backward()

Reading .grad

After backward(), each input tensor holds its gradient in .grad. For y = sum(x^2), the derivative is 2x, so the gradient is [4, 6].

print(x.grad)  # tensor([4., 6.])

Gradients Accumulate

A subtle gotcha: gradients add up across multiple backward() calls. In a training loop you must zero them each step with x.grad.zero_() or the optimizer's zero_grad().

x.grad.zero_()  # reset before the next backward()

torch.no_grad() for Inference

During inference you do not need gradients, and tracking them wastes memory. Wrap prediction code in with torch.no_grad(): to disable graph building and speed things up.

with torch.no_grad():
    preds = x.pow(2).sum()
    print(preds.requires_grad)  # False

detach() to Stop Gradient Flow

.detach() returns a tensor that shares data but is cut out of the computation graph. Use it to use a value without letting gradients flow back through it, for example logging or freezing part of a network.

z = x.detach()
print(z.requires_grad)  # False

Why Autograd Matters

Autograd is what makes deep learning practical: you define the forward computation, and PyTorch derives the gradients for backpropagation automatically. Every training loop you write rests on requires_grad, backward, and .grad.

Quick Check

Test your autograd understanding.

Recap: Tensors and Autograd

You created tensors with torch.tensor and enabled gradient tracking via requires_grad=True. You computed an output, ran .backward(), and read derivatives from .grad. You learned gradients accumulate, used torch.no_grad() for inference, and .detach() to cut gradient flow.

Frequently asked questions

Is the “PyTorch Tensors and Autograd” lesson free?

Yes — the full text of “PyTorch Tensors and Autograd” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.

What will I learn in “PyTorch Tensors and Autograd”?

torch.Tensor, requires_grad, backward(), gradient computation, tensor operations. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Learn AI with Python?

No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “PyTorch Tensors and Autograd” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Learn AI with Python lesson?

Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to Learn AI with Python