Expected object of device type cuda but got device type cpu in Pytorch

Just as a supplement or a general answer, each time you meet this cuda and cpu unmatched error, you should first check the following three things:

1. Whether you put your model on cuda, in other words, whether you have the similar code as:
   model = nn.DataParallel(model, device_ids=None).cuda()
2. Whether you put your input data on cuda, like input_data.cuda()
3. Whether you put your tensor on cuda, something like:
   loss_sum = torch.tensor([losses.sum], dtype=torch.float32, device=device)

Emm, if you do the three checks, maybe you will solve your problem, good luck.

In the forward of your MSE_loss, you define a linear layer that is probably still in the CPU (you didn't provide an MCVE, so I can only assume):

self.linear = nn.Linear(output1.size()[0], 1)

If you want to try and see if this is the problem, you can:

self.linear = nn.Linear(output1.size()[0], 1).cuda()

However, if self.d is in the CPU, then it would fail again. To solve this, you could move the linear to the same device of the self.d tensor by doing this:

def forward(self, output1, output2, labels):
    self.labels = labels         
    self.linear = nn.Linear(output1.size()[0], 1)

    if self.metric == 'cos':
        self.d = F.cosine_similarity(output1, output2)
    elif self.metric == 'l1':
        self.d = torch.abs(output1-output2)
    elif self.metric == 'l2':
        self.d = torch.sqrt((output1-output2)**2)

    # move self.linear to the correct device
    self.linear = self.linear.to(self.d.device)