Is calculating address difference undefined behaviour?

To quote the C++11 standard draft. On the subject of converting a pointer to an integer

[expr.reinterpret.cast]

5 A value of integral type or enumeration type can be explicitly converted to a pointer. A pointer converted to an integer of sufficient size (if any such exists on the implementation) and back to the same pointer type will have its original value; mappings between pointers and integers are otherwise implementation-defined.

Since uintptr_t must be defined for the your code to compile, then there exists an integer type on the target machine capable of being the target of the pointer-to-integer conversion. The mapping is implementation defined, but most importantly the result is not indeterminate. This means you obtain some valid integer for both conversions.

So the subtraction is not undefined behavior. But the result is implementation defined.

Converting pointer to integer of sufficient size is well defined, subtracting unsigned integer from another is well defined regardless of their value. There is no undefined behaviour here.

But also, standard doesn't guarantee any particular value for the converted integers, and therefore neither for the result of their subtraction.