When will Thread.Abort() not actually abort?

Did you even read the documentation?

The thread is not guaranteed to abort immediately, or at all. This situation can occur if a thread does an unbounded amount of computation in the finally blocks that are called as part of the abort procedure, thereby indefinitely delaying the abort. To wait until a thread has aborted, you can call the Join method on the thread after calling the Abort method, but there is no guarantee the wait will end.

...

If Abort is called on a thread that has not been started, the thread will abort when Start is called. If Abort is called on a thread that is blocked or is sleeping, the thread is interrupted and then aborted.

If Abort is called on a thread that has been suspended, a ThreadStateException is thrown in the thread that called Abort, and AbortRequested is added to the ThreadState property of the thread being aborted. A ThreadAbortException is not thrown in the suspended thread until Resume is called.

If Abort is called on a managed thread while it is executing unmanaged code, a ThreadAbortException is not thrown until the thread returns to managed code.

If two calls to Abort come at the same time, it is possible for one call to set the state information and the other call to execute the Abort. However, an application cannot detect this situation.

After Abort is invoked on a thread, the state of the thread includes AbortRequested. After the thread has terminated as a result of a successful call to Abort, the state of the thread is changed to Stopped. With sufficient permissions, a thread that is the target of an Abort can cancel the abort using the ResetAbort method. For an example that demonstrates calling the ResetAbort method, see the ThreadAbortException class.