Why doesn't running "sudo shutdown now" shut down?

Traditionally, the command sudo shutdown now will take you to the runlevel 1 (recovery mode); this will happen for both Upstart and SysV init. To get what you want, i.e., to shut down the computer properly, you need to give the -h switch to shutdown.

One thing to note here is that halt will close all the processes, turn off the CPUs and return the control to a ROM monitor of the mainboard needing the user to press the power button to get the power supply turned off, whereas poweroff after turning off the CPUs will simply turn off the power supply resulting in a proper shutdown.

The -h switch of shutdown will either halt or poweroff the computer, the decision will be taken by the system although in Ubuntu I have seen that it would normally poweroff the machine. To be sure of that, you can use the -P switch with shutdown to poweroff the computer.

To summarize, here are the commands available to poweroff (not halt) a computer:

sudo shutdown -h now
sudo shutdown -P now
sudo poweroff
sudo halt -p
sudo init 0

The poweroff and halt commands basically invoke shutdown (except for the poweroff -f). sudo poweroff and sudo halt -p are exactly like sudo shutdown -P now. The command sudo init 0 will take you to the runlevel 0 (shutdown).

Now what if you want to shut down forcefully, i.e., you don't want to wait for processes to close normally? In that case you can use:

sudo poweroff -f

This will not use shutdown. Rather, it will invoke the reboot(2) system call (used for reboot, poweroff & halt) to power off the computer instantly.

I'm pretty sure the correct command needs to contain the "-h" option (halt). Try this:

sudo shutdown -h now

It's a legacy from the days when the physical machine couldn't power off by itself. For example, the halt command sudo halt terminates all programs and unloads almost everything from RAM, ready to be powered off. However, if you run sudo halt -p, it will do all that, then signal the system to power off, or in the case of the shutdown command, you need the -a option, I believe, though I personally use halt.