Is HTML5 backwards compatible with XHTML?

First up, IE does indeed render intranet sites in compatibility mode by default. However, it is a config setting and can be switched off: if possible, I'd suggest that your first solution would simply be to switch off this setting on all the machines on your network. (whether this is a viable solution will depend on the size of your network, your group policies, and what other intranet sites would be affected)

Now to answer your question.

The HTML5 doctype can be specified on any site that you want to render in standards mode. If you're already using XHTML, the only effect for you will be that the browser will no longer rigidly enforce strict XHTML compliance. You are perfectly entitled to continue using XHTML code, but it won't be enforced.

The HTML5 doctype was specifically chosen because it works with existing browsers, including older versions of IE. You should be able to put it onto any page, and it should just work.

What it won't do is have any effect at all on IE going into compatibility mode. Nor will any other doctype. The X-UA-Compatible solution you mentioned is the solution to this (unless you can set the config setting I mentioned earlier), and is un-related to the doctype - you'll still need it even if you use the HTML5 doctype.

As far as I know, HTML5 is pretty much xhtml plus the extras so yeah, I think you can change the doctype and it would still work. They answered it here: If I use HTML 5's doctype, what will happen?

HTML5 was designed to be backwards compatible with both HTML 4.01 and XHTML 1.0/1.1, so that previous (X)HTML pages could be migrated to HTML5 easier. Of course, they wouldn't be taking advantage of new HTML5 features (such as new tags), but they would still be technically valid HTML5.