Learn AI Create the Future — with Pranay
Main site

Error database

RuntimeError: expected scalar type Float but found Double (PyTorch dtypes)

Two tensors in one operation have different dtypes — usually float64 data from NumPy meeting float32 model weights. Convert the input with .float(), and give loss functions the dtype they expect.

The message you saw
RuntimeError: expected scalar type Float but found Double (PyTorch dtypes)

By Updated

The error

Output
RuntimeError: expected scalar type Float but found Double

The pair of type names varies with the collision:

Output
RuntimeError: expected scalar type Double but found Float
RuntimeError: expected scalar type Long but found Float
RuntimeError: Expected floating point type for target with class probabilities, got Long

What it means

Every tensor has a dtype — the numeric format of its elements. Float is float32, Double is float64, Long is int64. PyTorch refuses to mix dtypes silently in most operations, because silent conversion hides bugs and wastes memory. One of your tensors is in the wrong format for the operation.

Why it happens

Float/Double collisions come from NumPy. NumPy's default float is float64. PyTorch models default to float32. So torch.from_numpy(X) produces a Double tensor, and the first linear layer rejects it.

Long/Float collisions come from loss functions and embeddings. CrossEntropyLoss wants targets as Long class indices. MSELoss wants Float values. nn.Embedding wants Long indices. Feeding the wrong one produces the Long/Float family of messages.

How to fix it

1. Convert NumPy-derived inputs to float32 once, at creation.

python
X = torch.from_numpy(X_np).float()          # float64 -> float32
# or
X = torch.tensor(X_np, dtype=torch.float32)

Do this in the Dataset's __getitem__ or right after loading — one place, not scattered through the code.

2. Give each loss function its expected target dtype.

python
# classification: class indices, Long
loss = nn.CrossEntropyLoss()(logits, targets.long())

# regression: values, Float
loss = nn.MSELoss()(preds, targets.float())

3. Keep embedding and index inputs Long.

python
token_ids = torch.tensor(ids, dtype=torch.long)
emb = embedding_layer(token_ids)

4. Do not fix it by converting the model to double. model.double() makes the error go away while doubling memory and slowing the GPU, for no accuracy benefit in normal training. Convert the data down, not the model up.

5. When lost, print both dtypes at the failing line.

python
print(x.dtype, next(model.parameters()).dtype)

How to prevent it

Establish the convention early: inputs float32, class targets long, regression targets float32 — enforced inside the Dataset so every batch arrives correct. Any .astype(np.float64) or bare np.array on the way into PyTorch is a Double waiting to happen.