System.IO.File.Move gives Exception "The file exists"

This comes out of Windows, different error codes for different scenarios. The second one is ERROR_ALREADY_EXISTS, "Cannot create a file when that file already exists" when the file is moved from one directory to another on the same drive. That's a pretty simple operation, it only requires moving the directory entry.

The first one is ERROR_FILE_EXISTS, "The file exists" when the file is moved from one drive to another drive. That's a much more involved operation, the file data has to be copied as well. In other words, it falls back to the equivalent of File.Copy(string, string, bool) with the last overwrite argument set to false. That it doesn't use the same error code is a bit of a quirk. The difference is a big deal to the file system driver, just not to your program. Otherwise the reason that you just get a pretty generic IOException instead of a more specific one that breaks down the file manipulation mishaps into finer-grained exceptions.

It isn't actually a problem because there isn't anything you can do about it in your code, you need the help of a human to correct the problem. Unless you take specific pre-emptive measures in your own code, either avoiding the move if the destination file already exists or by actually deleting the destination file first. Do note that neither is a 100% reliable workaround, there's a very small chance that another process creates the file again right after you deleted it but before you move. Making file operations completely reliable is pretty difficult on a multi-tasking operating system.