Can I stop all processes using CUDA in Linux without rebooting?

You can use the fuser command to get the all the processes using CUDA and then kill them. There's also a nice single command to kill them all.

sudo fuser -k /dev/nvidia*

you can check the processes with nvidia-smi and then

kill -9 <pid>

The lsof utility will help with this. You can get a list of processes accessing your NVIDIA cards with:

lsof /dev/nvidia*

Then use kill or pkill to terminate the processes you want. Note that you may not want to kill X if it's running. On my desktop system, both X and kwin are also accessing the GPU.

Long answer:

lsof /dev/nvidia*

gives you PIDs running on your GPU card which looks something like: lsof: status error on PID: No such file or directory

COMMAND  PID    USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
python  7215 *******  mem    CHR 195,255           434 /dev/nvidiactl
python  7215 *******  mem    CHR   195,0           435 /dev/nvidia0

and

awk '{print $2}'

selects the PID column (in my case it is the second column) and

xargs -I {} kill {}

kills those PID jobs.

Short answer:

You may use the following command to remove them all at once.

Watch out! This command will delete all PIDs showing up for lsof /dev/nvidia*. Do run lsof /dev/nvidia* first to confirm these jobs are the ones you want to delete.

lsof /dev/nvidia* | awk '{print $2}' | xargs -I {} kill {}

Finish the job by a single command.