Why is IdentityMatrix not defined as a SparseArray by default?

The fastest way to get the identity matrix as a sparse array is simply this:

IdentityMatrix[10000, SparseArray]; // AbsoluteTiming

Here are just some thoughts on why SparseArray is not the default for IdentityMatrix:

You don't always want SparseArray output when defining matrices, and it's not always possible to decide automatically whether a SparseArray should be cast into List form (simple example: TraditionalForm or other displays). Therefore, when SparseArray was introduced, it could not simply replace the existing representation of arrays as lists. In particular, this holds for IdentityMatrix which is after all of use not only in high-dimensional spaces but also at low dimensionality where SparseArray doesn't represent the most efficient way of storing and manipulating arrays. For more potential reasons justifying the default "non-sparseness" of IdentityMatrix, see the comments.

SparseArray objects are somewhat special and some of the access operations can be slower with them. IdentityMatrix though does return a more compact object than just a list of lists -- it returns a packed array.

Needs["Developer`"]

Tally[
 Table[PackedArrayQ[IdentityMatrix[n]], {n, Range[1, 1000, 10]}]]

(* {{True, 100}}*)

Also, it is really easy to covert the matrices of IdentityMatrix into sparse arrays:

Table[SparseArray[IdentityMatrix[n]], {n, Range[100, 1000, 100]}]