Is it allowed to cast away const on a const-defined object as long as it is not actually modified?

Yes. This is entirely legal. (It is dangerous, but it is legal.) If you (attempt to) modify a an object declared const, then the behaviour is undefined.

From n4659 (which is the last draft of C++17), section 10.1.7.1 [dcl.type.cv] para 4:

Except that any class member declared mutable (10.1.1) can be modified, any attempt to modify a const object during its lifetime (6.8) results in undefined behavior

My emphasis. That is from C++17, but this has been true of all versions of C++.

If you look at the section on const_cast there is a note that

[ Note: Depending on the type of the object, a write operation through the pointer, lvalue or pointer to data member resulting from a const_cast that casts away a const-qualifier76 may produce undefined behavior (10.1.7.1). — end note ]

Notes are not normative, but this strongly implies that obtaining a non-const reference or pointer to a const object is legal. It is the write that is not allowed.