Qubits of higher dimension

Qubits are the standard 'currency' in quantum information theory for the same reason that bits are in classical information theory. It is because they suffice for the treatment of many of the main ideas.

When it comes to practical considerations, then in the lab one uses whatever systems one can control most easily. In ion trap work, for example, it is very common to use more than two of the internal states of each ion to do some of the work of storing and processing information, but nonetheless it usually happens that just two per ion are best able to be protected against magnetic field noise and things like that. The move from 2 to $k$ states per entity makes a saving in the number of entities (e.g. atoms or flux loops or whatever) required to store a given body of information, but this is accompanied by an increase in the complexity of the operations required to manipulate those entities. For this reason the move from qubits to qutrits is not employed much (believe me, if it were a useful move then experimentalists would have made it immediately!)

One might also suggest using a very large number of internal states of a given entity, but then the control problem becomes insuperable. It is a very important property of quantum states that if one has a collection of $n$ qubits stored in $n$ separate systems then the sensitivity to noise scales only in proportion to $n$ or possibly $n^2$, whereas if one tries to use $2^n$ states of a single thing (e.g. internal energy levels) then the sensitivity to noise scales as $2^n$. This is one of the wonderful features of entanglement and lies close to the heart of why quantum computing is powerful.