Model is not learning: problem with custom activation function and/or with custom loss function

Question

class QuaternionLoss(torch.nn.Module):
    def __init__(self):
        super(QuaternionLoss, self).__init__()
        
    def forward(self, output, target):
        loss = 100 * (1 - torch.dot(output.squeeze(0), target.squeeze(0)))
        return loss

class LinearNormalized(torch.nn.Module):
    def __init__(self):
        super(LinearNormalized, self).__init__() # init the class 
    
    def forward(self, x):
        return linear_normalized(x)

class VGGOrientation(torch.nn.Module):
    def __init__(self):
        super(VGGOrientation, self).__init__()
        self.model_vgg_orientation = torchvision.models.vgg16(pretrained=True)
        self.model_vgg_orientation.classifier = torch.nn.Sequential(
            torch.nn.Linear(25088, 256),
            torch.nn.ReLU(inplace=True),
            torch.nn.Linear(256, 64),
            torch.nn.ReLU(inplace=True), 
            torch.nn.Linear(64, 2),
            LinearNormalized()
        )

    def forward(self, x):
        output_orientation = self.model_vgg_orientation(x)
        return output_orientation

When I train my model this is what I have ( I printed out the loss function, my target and the output of my model): enter image description here

Do you have any idea on why my model is not learning? Something has to be wrong with the activation function or the loss because my data is good. In fact I have created the same model on TensorFlow and it works.