Why is TypedReference behind the scenes? It's so fast and safe... almost magical!

Short answer: portability.

While __arglist, __makeref, and __refvalue are language extensions and are undocumented in the C# Language Specification, the constructs used to implement them under the hood (vararg calling convention, TypedReference type, arglist, refanytype, mkanyref, and refanyval instructions) are perfectly documented in the CLI Specification (ECMA-335) in the Vararg library.

Being defined in the Vararg Library makes it quite clear that they are primarily meant to support variable-length argument lists and not much else. Variable-argument lists have little use in platforms that don't need to interface with external C code that uses varargs. For this reason, the Varargs library is not part of any CLI profile. Legitimate CLI implementations may choose not to support Varargs library as it's not included in the CLI Kernel profile:

4.1.6 Vararg

The vararg feature set supports variable-length argument lists and runtime-typed pointers.

If omitted: Any attempt to reference a method with the vararg calling convention or the signature encodings associated with vararg methods (see Partition II) shall throw the System.NotImplementedException exception. Methods using the CIL instructions arglist, refanytype, mkrefany, and refanyval shall throw the System.NotImplementedException exception. The precise timing of the exception is not specified. The type System.TypedReference need not be defined.

Update (reply to GetValueDirect comment):

FieldInfo.GetValueDirect are FieldInfo.SetValueDirect are not part of Base Class Library. Note that there's a difference between .NET Framework Class Library and Base Class Library. BCL is the only thing required for a conforming implementation of the CLI/C# and is documented in ECMA TR/84. (In fact, FieldInfo itself is part of the Reflection library and that's not included in CLI Kernel profile either).

As soon as you use a method outside BCL, you are giving up a bit of portability (and this is becoming increasingly important with the advent of non-.NET CLI implementations like Silverlight and MonoTouch). Even if an implementation wanted to increase compatiblility with the Microsoft .NET Framework Class Library, it could simply provide GetValueDirect and SetValueDirect taking a TypedReference without making the TypedReference specially handled by the runtime (basically, making them equivalent to their object counterparts without the performance benefit).

Had they documented it in C#, it would have had at least a couple implications:

1. Like any feature, it may become a roadblock to new features, especially since this one doesn't really fit in the design of C# and requires weird syntax extensions and special handing of a type by the runtime.
2. All implementations of C# have to somehow implement this feature and it's not necessarily trivial/possible for C# implementations that don't run on top of a CLI at all or run on top of a CLI without Varargs.

Well, I'm no Eric Lippert, so I can't speak directly of Microsoft's motivations, but if I were to venture a guess, I'd say that TypedReference et al. aren't well documented because, frankly, you don't need them.

Every use you mentioned for these features can be accomplished without them, albeit at a performance penalty in some cases. But C# (and .NET in general) isn't designed to be a high-performance language. (I'm guessing that "faster than Java" was the performance goal.)

That's not to say that certain performance considerations haven't been afforded. Indeed, such features as pointers, stackalloc, and certain optimized framework functions exist largely to boost performance in certain situations.

Generics, which I'd say have the primary benefit of type safety, also improve performance similarly to TypedReference by avoiding boxing and unboxing. In fact, I was wondering why you'd prefer this:

static void call(Action<int, TypedReference> action, TypedReference state){
    action(0, state);
}

to this:

static void call<T>(Action<int, T> action, T state){
    action(0, state);
}

The trade-offs, as I see them, are that the former requires fewer JITs (and, it follows, less memory), while the latter is more familiar and, I would assume, slightly faster (by avoiding pointer dereferencing).

I'd call TypedReference and friends implementation details. You've pointed out some neat uses for them, and I think they're worth exploring, but the usual caveat of relying on implementation details applies—the next version may break your code.

I can't figure out whether this question's title is supposed to be sarcastic: It has been long-established that TypedReference is the slow, bloated, ugly cousin of 'true' managed pointers, the latter being what we get with C++/CLI interior_ptr<T>, or even traditional by-reference (ref/out) parameters in C#. In fact, it's pretty hard to make TypedReference even reach the baseline performance of just using an integer to re-index off the original CLR array every time.

The sad details are here, but thankfully, none of this matters now...

This question is now rendered moot by the new ref locals and ref return features in C# 7

These new language features provide prominent, first-class support in C# for declaring, sharing, and manipulating true CLR managed reference type-types in carefully prescibed situations.

The use restrictions are no stricter than what was previously required for TypedReference (and the performance is literally jumping from worst to best), so I see no remaining conceivable use case in C# for TypedReference. For example, previously there was no way to persist a TypedReference in the GC heap, so the same being true of the superior managed pointers now is not a take-away.

And obviously, the demise of TypedReference—or its nearly complete deprecation at least—means throw __makeref on the junkheap as well.