Arbitrary-Precision Math in PHP

GMP is a ton faster BCMath, although BCMath can be made faster using OpenSSL. Here's a benchmark comparing the various techniques:

http://phpseclib.sourceforge.net/math/intro.html

what exactly is arbitrary-precision math?
Arbitrary precision arithmetic aka "bignum math", introduces a way of performing arithmetic operations on numbers which number of digits are only limited by the amount of memory available. This is in departure with the fixed precision arithmetic which is afforded by the CPUs/ALUs of the host systems and where the maximum size/precision of the number represented is a factor of the number of bits of the registers of these hardware processors.

Fixed precision arithmetic is fast, efficient with regards to storage and it is built-in/universally available. It is however applicable to limited (if only sometimes "big enough") numerical ranges. Arbitrary precision arithmetic is slower, somewhat wasteful of the storage and requires specialized libraries such as GMP or BCMath.

what are the differences between the BCMath and GMP libraries
The most salient difference is that GMP works on [arbitrary precision] integer values, whereby BCMath allows [arbitrary precision] decimal / float-like values.
Neither API is hard to learn, but BCMath may be a bit more intuitive (in addition to support float-like values)

One's selection of a particular library over another one is typically driven by the intended use (or by the availability on a given platform). Until you get heavily into MP applications, most library will fit the bill and be generally equivalent (within its class of course, i.e. avoid integer-only library if you need floating point numbers).

what type of numbers BCMath/GMP takes?
As with most arbitrary precision math packages, these two libraries use strings for their API, i.e. to represent their input and output numeric values.
Internally... Some packages like GMP have their own representation for the numbers. The specific of such structures is typically a compromise between minimizing storage requirements and allowing fast computations (including that of "serializing/deserializing" such structures to/from text files.)
The example "\x12\x23\x45\x67" in the question is known as BCD i.e. Binary Coded Decimal. It allows storing 2 decimal digits per byte and is sometimes used by Arbitrary Precision Arithmetic libraries.