class crossentropy(nn.Module):
def __init__(self):
super(crossentropy, self).__init__()
def forward(self, y_1, y):
m = nn.Softmax(dim=1)
output = m(y_1)
loss = -1.0*torch.sum(y*torch.log(output))
l = torch.mean(loss)
return l
Asked
Active
Viewed 348 times
0
Mateen Ulhaq
- 24,552
- 19
- 101
- 135
abhilash sharma
- 11
- 2
-
There are two possible cases, either some elements of y or y_1 are NaN, or elements of output are too close to 0 – tbrugere Dec 16 '21 at 00:29
-
In any case, you should use LogSoftmax instead of taking the log of the softmax, that may avoid the 2nd case – tbrugere Dec 16 '21 at 00:29
-
`m` should be created in `__init__` as `self.m = ...`. – Mateen Ulhaq Dec 16 '21 at 00:31
-
Have you tried replacing your `crossentropy` class with the standard classes and seeing if that still doesn't work? – Mateen Ulhaq Dec 16 '21 at 00:33
1 Answers
0
It is probable that using LogSoftmax instead of Softmax then log should solve this issue (probably caused by the log giving infinite values to near-0 results of softmax, due to numerical error)
tbrugere
- 755
- 7
- 17
-
class crossentropy(nn.Module): def __init__(self,m=0): super(crossentropy, self).__init__() self.m = m def forward(self, y_1, y): m = self.m m = nn.LogSoftmax() output = m(y_1) loss = -1.0*torch.sum(y*output) l = torch.mean(loss) return l – abhilash sharma Dec 16 '21 at 03:48
-
It still gives the same error, I tried removing log and it gives good accuracy score and negative loss – abhilash sharma Dec 16 '21 at 03:52
-