Can I use two types of DDR3 in the same motherboard?

Yes. As long as both modules are supported by the board, the faster will clock down to match the slower. Enjoy your 8GB of ram.

Also, you don't have to worry too much about breaking your motherboard as long as the memory types matches the slot. At worst it'll fail to POST. Google 'Power On Self Test' for more.

I have a DDR3 RAM of 1333MHz of 4GB on my motherboard (Asus H61M). Can I use a DDR3 1600 MHz of 4GB at the another slot?

Yes

The faster memory should be automatically down clocked to 1333Mhz. Be sure you are running the current firmware to avoid any issues ahead of time.

The motherboard, and I would assume your CPU, supports both speeds. You could in theory over clocked the slower memory, although that might not be worth your time since a 267Mhz down clock won't result in any performance differences honestly.

Each DDR module has a special chip called SPD. This chip contains information about supported modes (frequency and latencies this module should work with).

Memory controller situated in modern CPUs reads this information and chooses best mode, supported by all installed modules. While it works with all modules simultaneously it can't use different freq or latency for different modules.

This behaviour could be changed by Setup program (people call it "bios setup" sometimes). Overclockers set freq and latency manually, overwriting values from SPD, but system may be unstable.

In your case memory controller will read all SPDs and choose the slowest frequency. It will work. You may try to set big frequency manually, but you may face BSOD or even unbootable PC then, and you will need to reset memory settings.

After installing DRAM, use CPU-Z tool (it is free, google for it) to read freq, latencies and other values your controller set. It also can show you SPD values.