How can I monitor internet usage in my network?

If you want to graphically display your local network connections, you may be interested in etherape. Free and cross platform. The thickness of the connection is an indicator of traffic volume.

At the router level would be the easiest as it wouldn't involve installing software on each machine separately. Tomato Firmware is great and automatically monitors bandwidth in the logs section, you can use it along with IPTables (Linux) for a complete traffic monitoring solution. Since you are probably on Windows, you can use it in a VM as well.

For a software solution for each computer, take a look at the freeware tool NetLimiter, the bandwidth monitor is free:

The Windows Resource Monitor on 7 (and Vista?) has grown quite advanced in what it shows under "network".