Calling backward on a non-scalar output

Setup#

pip install torch

Outputs verified with torch 2.5.1, CPU.

The refusal, and the two fixes#

import torch

x = torch.tensor([1., 2., 3.], requires_grad=True)
y = x * x                       # y holds 3 numbers, not 1

try:
    y.backward()
except RuntimeError as err:
    print("refused:", err)

y.backward(torch.ones_like(y))  # weight every output equally
print(x.grad)                   # 2x, same as y.sum().backward()

x.grad = None                   # clear before the next experiment
y = x * x
y.backward(torch.tensor([1., 0., 0.]))   # care ONLY about y[0]
print(x.grad)

refused: grad can be implicitly created only for scalar outputs
tensor([2., 4., 6.])
tensor([2., 0., 0.])

The walkthrough#

The error names the rule. "Implicitly created" refers to the starting blame: for a scalar, PyTorch seeds the walk with 1.0 automatically. For a vector it will not invent the seed — the argument you pass is that seed.

All-ones gave [2., 4., 6.]. Each y_i is x_i squared and touches only its own x_i, so each input's blame is its own derivative, 2x. Equal weighting reproduces exactly what y.sum().backward() computes — in practice, write the .sum(): clearer intent, same numbers, same speed.

[1., 0., 0.] gave [2., 0., 0.]. Zero weight, zero blame. The seed vector lets you ask about one output of many — this is the mechanism, exposed. What backward computes in general is the weighted combination "seedᵀ times the table of all output-input sensitivities" — the researcher block makes that precise.

The x.grad = None line matters. Gradients accumulate, and without clearing, experiment two would print the sum of both experiments' answers. When your interactive results look mysteriously doubled, this is why.

The place you meet this in real life#

Per-sample losses. F.mse_loss(pred, target, reduction="none") returns one loss per sample — a vector. Calling backward on it raises this exact error, and the fix is choosing your weighting consciously: .mean() for the usual average, or a weighted sum when samples deserve different importance:

import torch
pred = torch.tensor([2., 5., 9.], requires_grad=True)
target = torch.tensor([1., 5., 7.])
per_sample = (pred - target) ** 2
weights = torch.tensor([1., 1., 4.])     # third sample counts four times
(per_sample * weights).sum().backward()
print(pred.grad)

tensor([ 2.,  0., 16.])

The third sample's gradient is 2·(9−7)·4 = 16 — importance weighting, dressed as the backward seed.

Common mistakes#

Reflexively writing .sum() to silence the error. It works, but decide between sum and mean: mean divides gradients by the batch size, so learning rates tuned for one do not transfer to the other. Changing reduction mid-project changes your effective learning rate. See loss functions.

Believing y.backward(v) gives "the gradient of y". A vector-valued y has no single gradient — it has a table (one row per output). What you get is v's weighted combination of rows. For the full table, the tools are different: jacrev and friends.

Shape mismatch in the seed. The seed must match y's shape exactly; a wrong-shaped seed raises a mismatch error naming "grad_output". Build it with torch.ones_like(y) and reshape from there.

Forgetting the seed exists when reading others' code. loss.backward() is secretly loss.backward(torch.tensor(1.0)). Knowing that makes gradient scaling tricks — like multiplying the loss by a constant — stop looking magical: the constant rides along on every gradient.

Try it yourself#

With y = x * x and x = [1., 2., 3.], first predict x.grad for the seed [0., 1., 0.], then for [2., 2., 2.]. Check both, then confirm the second equals what (2 * y.sum()).backward() produces — and explain why in one sentence.

What to learn next#

• Jacobians, vjp and jvp with torch.func — when you genuinely need the whole table.
• In-place operations and when they break autograd — the other classic backward-time error.
• Loss functions — choosing the scalar that deserves the blame.