Wanted: Command line HTML5 beautifier
HTML Tidy has been forked by the w3c and now has support for HTML5 validation.
https://github.com/w3c/tidy-html5
I suspect tidy can be made to work with the right command-line parameters.
http://tidy.sourceforge.net/docs/quickref.html
You can specify an arbitrary doctype and add new block, inline, and empty tags, and turn on and off lots of tidy's cleaning options.
Depending on what you want it to "beautify" you can probably get decent results. It probably won't be able to do some of the more advanced things like rewriting the html content to eliminate spurious elements or combining them, if it doesn't recognize them.