123

I'm trying to train a classifier via PyTorch. However, I am experiencing problems with training when I feed the model with training data. I get this error on y_pred = model(X_trainTensor):

RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #4 'mat1'

Here are key parts of my code:

# Hyper-parameters 
D_in = 47  # there are 47 parameters I investigate
H = 33
D_out = 2  # output should be either 1 or 0

# Format and load the data
y = np.array( df['target'] )
X = np.array( df.drop(columns = ['target'], axis = 1) )
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size = 0.8)  # split training/test data

X_trainTensor = torch.from_numpy(X_train) # convert to tensors
y_trainTensor = torch.from_numpy(y_train)
X_testTensor = torch.from_numpy(X_test)
y_testTensor = torch.from_numpy(y_test)

# Define the model
model = torch.nn.Sequential(
    torch.nn.Linear(D_in, H),
    torch.nn.ReLU(),
    torch.nn.Linear(H, D_out),
    nn.LogSoftmax(dim = 1)
)

# Define the loss function
loss_fn = torch.nn.NLLLoss() 

for i in range(50):
    y_pred = model(X_trainTensor)
    loss = loss_fn(y_pred, y_trainTensor)
    model.zero_grad()
    loss.backward()
    with torch.no_grad():       
        for param in model.parameters():
            param -= learning_rate * param.grad
prosti
  • 42,291
  • 14
  • 186
  • 151
Shawn Zhang
  • 1,719
  • 2
  • 14
  • 20

9 Answers9

172

Reference is from this github issue.

When the error is RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #4 'mat1', you would need to use the .float() function since it says Expected object of scalar type Float.

Therefore, the solution is changing y_pred = model(X_trainTensor) to y_pred = model(X_trainTensor.float()).

Likewise, when you get another error for loss = loss_fn(y_pred, y_trainTensor), you need y_trainTensor.long() since the error message says Expected object of scalar type Long.

You could also do model.double(), as suggested by @Paddy .

MilkyWay90
  • 2,023
  • 1
  • 9
  • 21
57

Before converting to Tensor, try this

X_train = X_train.astype(np.float32)
Mateen Ulhaq
  • 24,552
  • 19
  • 101
  • 135
Shubham Mishra
  • 834
  • 6
  • 8
8

The issue can be fixed by setting the datatype of input to Double i.e torch.float32

I hope the issue came because your datatype is torch.float64.

You can avoid such situations either while setting the data, as explained in one of other answers or make the model type also to the same as of your data. i.e use either float64 or float32.

For debug, print obj.dtype and check for consistency.

Rohit Davas
  • 121
  • 1
  • 3
3

Let's do that:

df['target'] = df['target'].astype(np.float32)

and for x features too

Wojciech Moszczyński
  • 2,893
  • 21
  • 27
1

This issue can also occur if the wrong loss function is selected. For example, if you have regression problem, but you are trying to use cross entropy loss. Then it will be fixed by changing your loss function on MSE

Dmitriy Ershov
  • 31
  • 4
1

try to use: target = target.float() # target is the name of error

Paradigm
  • 452
  • 5
  • 9
1

New to PyTorch. For some reason, calling torch.set_default_dtype() with the needed datatype was what worked for me on Google Colab. network.double()/network.float() and tensor.double()/tensor.float() didn’t have any effect, for some reason.

mayowa_osibodu
  • 56
  • 6
0

Building off this answer, to convert all columns of one type to another...

df.update(df.select_dtypes("float64").astype("float32"))

Note that this is an in-place operation.

David Gilbertson
  • 4,219
  • 1
  • 26
  • 32
-1

Try this example:

from sentence_transformers import SentenceTransformer, util
import numpy as np
import torch

a = np.array([0, 1,2])
b = [[0, 1,2], [4, 5,6], [7,8,9]]

bb = np.zeros((3,3))
for i in range(0, len(b)):
    bb[i,:] = np.array(b[i])


a = torch.from_numpy(a)
b = torch.from_numpy(bb)

a= a.float()
b = b.float()

cosine_scores = util.pytorch_cos_sim(b, a)
print(cosine_scores)
Farzad Amirjavid
  • 649
  • 5
  • 13