What exactly are the differences between a USB host and device

What exactly are the differences between a USB host and device?

The host initiates all communication on the bus, the device only responds when asked by the host. For Details see the specs on usb.org.

When two devices connect [...]

One must behave as the host and the other as the device. Details can be found in the USB OTG Specs.

Can one device acts as a host as well as a device?

For example, many Android phones and tablets can (requires Android 3+). The "magic" is in the On-the-Go cable/adapter, which switches the phone into host mode by pulling the ID pin to GND.

Just like Stéphane, I came here for the answer. Which I did not find here! With a little research this is what I found.

With a host controller you will be able to communicate with all USB devices, and with a USB device controller you can just communicate with a host controller.

USB host is the USB on the PC side in most cases and USB Device is the USB in your mouse ,keyboard, flash memory and so on.

All USB transactions are managed by the Host. and the Device only responses to the Host transactions.

Hope that helps.