To use or not to use the ObjectManager directly?

You should not use the ObjectManager directly!

Exception from the rule are:

• in static magic methods like __wakeup, serialize, etc
• in case you should make backward compatibility of constructor
• in global scope, like in fixtures of integration test.
• in class that needs only for creation of object like factory, proxy , etc

You should never use \Magento\Framework\App\ObjectManager::getInstance().
It defeats the purpose of dependency injection. We're back at Mage::getModel().
Object manager should be used only in factories and then as injected in a constructor.

The advantage of using this is less code to write. But this does not make it OK.
The fact that this is still used in the core, is because it didn't get refactored yet. I hope it will be.

So why does M2 sometimes access object manager directly when we recommend against it?

Brutal answer: M2 is a port of M1 - not a complete rewrite. So don't assume that all the M2 code is perfectly ported yet (unfortunately). Just because you find something in the M2 code base, that does not mean "its the best way to do it". Sometimes it is just "we have not got around to fixing it yet".

Less brutal: As per other responses, sometimes you MUST use it as there is no alternative. Other times it might be for backwards compatibility reasons. And framework code sometimes makes sense using it directly, because it is framework code. But if I had to guess without looking at code, many really should be fixed but it has not been high enough priority to do so yet.

Just remember the good parenting advice: "Kids, do what I say, not what I do!"