How do I debug an upstart job?

You can redirect stdout and stderr of the entire shell using the script pragma (instead of exec) in conjunction with exec >FILE 2>&1, like so:

script
    exec >/path/to/some_log_file 2>&1
    exec your_command_here
end script

That should hopefully give you better insight into what's going on. I've found this useful for catching all sorts of problems in my upstart scripts. You could pipe your command's stdout/stderr directly, but you'll miss out on errors originating in the shell (like syntax errors).

On the other hand, if service is hanging, it might not even be hitting your script, in which case none of this will help, of course.

There's also the console log declarative, as defined here: http://upstart.ubuntu.com/cookbook/#console-log

I don't know enough about upstart to know if it's enabled by default, but you can enable it on a per upstart job basis, it'll by default output to /var/log/upstart/<job>.log