Invalid device Ordinal , CUDA / TORCH

Try doing this

import torch
print(torch.cuda.is_available())

If you get the output to be False, that means PyTorch hasn't detected the GPU. I had the same issue and reinstalling Pytorch worked for me. You might also want to look at this https://github.com/pytorch/pytorch/issues/6098 .