Which programs are using my USB flash drive?

This tool and this tool both look like they would serve the purpose you are trying to achieve. The NirSoft interface is a little less direct, but I trust the author and his ability to write code. The second seems to have a more useful UI in your situation, but I have no history with the website or author.

ProcessExplorer is another easy way to do it, simply open up ProcessExplorer, hit Ctrl+F, and then type the name of the file. ProcessExplorer will list all the programs using that file. If the WhoLockMe tool doesn't work for you, I'd recommend ProcessExplorer next, simply because it has so many other useful features.

I use EjectUSB (working download link):

EjectUSB could be considered the nuclear option of USB drives that just won't properly eject in Windows, because there's an "application or process" accessing it. Put EjectUSB on your thumb drive and run it, and the program will mercilessly kill every program, process, or anything else touching your drive, letting you safely remove it without any fear of data loss.

In the Event Viewer (checked on windows 10) -> Windows Logs -> System - you can see what happened.

Example:

The application \Device\HarddiskVolume1\Program Files\Security\AVAST Software\Avast\AvastSvc.exe with process id 2080 stopped the removal or ejection for the device USB\VID_0BC2&PID_2300\2GH4ER2Z____.