How to know if my webcam is used or not?

If your kernel uses modules (which is highly likely), one way to determine whether a program is accessing your webcam is to look at the usage count of the module:

$ lsmod | grep uvcvideo
uvcvideo               90112  0

The 0 in the third field shows that nothing has any device open for a uvcvideo-controlled webcam (when lsmod ran). Of course you need to know exactly which module is responsible for your webcam; it's easy to check though, you'll see the output change while running a program such as Cheese.

Note that, strictly speaking, a positive count only means that something has opened a device, it doesn't mean images are being captured.

On any sane system, unless you have set up chroots with their own /dev, all device files are under /dev. Only root can create device files, so you don't need to worry about malicious users creating device files elsewhere.

So all you need to do is locate the files under /dev that refer to the same device as the one you're interested in.

ls -lR /dev |awk '/^c/ && $5 == "81," && $6 == "0"'

It's likely that this will show only /dev/video0. Usually, there's a single device file for each device, and there are possibly additional symbolic links to it.

Thus the practical answer to your question is the simple one. Just check what processes have the device file open.

fuser /dev/video0

If you want to monitor accesses (i.e. catch processes that may open the device file at any time), use one of Linux's file access monitoring methods on the device file(s): set up a watch (and check what processes already have the device file(s) open)

inotifywait -m -e open,close /dev/video0 &
sleep 1; fuser /dev/video0   # check for processes that have already opened the device

or set up an audit rule which will log accesses in the system logs (typically /var/log/audit/audit.log)

auditctl -w /dev/video0 &
sleep 1; fuser /dev/video0   # check for processes that have already opened the device

Assuming that what you actually want is to make sure your webcam isn't being used when you don't want it to, the simplest solution is to simply disconnect it (if external) when not needed. Or covering the webcam (just a piece of duct tape would work).

Physically-based approaches are much more secure than software ones.