Alternatives to using Thread.Sleep for waiting

Well, you said most of it. Quoting "We all know thread creation is expensive and blocking in threads means contention on the pool", so you even understand what using a pool of threads is about.

You also understand that blocking the UI thread is bad.

Looking at the pool of threads model again: you have a pool of threads, maybe one per processor, and then pass tasks to them. What sense would it make for one of those threads to block? If it does not have work to perform now, then it should simply proceed to a different task.

So, going directly to your question "Which leads me to my point, if you really need to perform a sleep, what should you be using if not Thread.Sleep?", in a modern well designed program, you would never need to do it, you would simply schedule a task for latter.

You should think of the threads in a pool, much as the processors in a system, as resources, which should be released to others when not needed.

Going to your examples, you're a bit too involved in the imperative programming paradigm.

• You don't need to wait for a process to disappear... I don't imagine why you need this, but if have to wait, it's because you have work to perform after sometime, the "continuation" of your function. You should set up a timer for this "continuation".
• The file examples should have other mechanisms for it, if they do not have.... that would be good OS design. For instance, the only safe way to wait for buffers to flush would be an OS primitive, on the likes of fsync.
• If there is someone writing to a file and then another reading from the file, then a synchronization mechanism is needed, not a timed wait (unless the file is append-only, in which case the file itself is the synchronization mechanism).

Waiting on a synchronization mechanism is not "bad".

The WaitHandle type and derived types provide an event-driven mechanism for waiting that ties into the operating system. For example, when you have a Task<T> task and you wait on the result by accessing task.Result, the internal implementation isn't polling with Thread.Sleep calls in between. It's using a WaitHandle-derived type to do waiting and synchronization.

Sometimes a polling-based approach is necessary, as in some of the examples you gave in your bullet list, but often you can use an event-driven approach instead. It's not that Thread.Sleep is always bad - it's just that it is very often misused.

It is just the nature of software to wait for other things in a multi-threaded environment with external factors affecting your code, sometimes you need to wait...

To wait is fine. To wait with polling is often not (*). If there is any way you can use an event-driven wait, you should typically strive to use that.

I don't have a very good feel for exactly what it is that you're asking, so I won't elaborate beyond this. If you leave a comment I can expand my answer.

---

(*) The theoretical reason waiting with polling is bad is as follows:

Suppose I have code that looks like this:

//START
Begin();
while (!Done())
    Thread.Sleep(D);
//STOP

Begin() starts some operation. Done() returning true means the operation has finished. Suppose this will happen after approximately T time. Then:

• The thread wakes up and checks the condition (calls Done()) T/D times
• The duration from START to STOP includes an expected D/2 purely because of the Thread.Sleep

What value of D should you choose? As you increase D, the expected duration form START to STOP increases linearly. As you decrease D, the (bound on the) number of iterations increases as 1/D. Both of these are bad, and finding the right D is problematic.

Now compare this to an event-driven wait:

//START
Begin();
WaitDone();
//STOP

Theoretically speaking, as long as WaitDone() somehow magically waits until the operation has finished but no longer, both of the problems identified in the waiting with polling case have disappeared: this thread waits for exactly the right amount of time - no more, no less!

To reiterate the point I started with: in .NET, the WaitHandle class and derived types are what facilitate this approach.