What's the use of a finally block preceded by a catch-all catch block, in C#?

The assumption that S1 cannot throw is a fragile one, considering resource depletion scenarios (i.e., you run out of memory). Even if warranted (a big if), minor changes to the code can introduce an exception.

Since S2 is usually concerned with cleaning up and releasing valuable resources, putting it in a finally block communicates that intention clearly. Putting such code, where possible, in a Dispose() method of a resource owning object and replacing the try/finally clause with a using clause can communicate the intention even better (and more idiomatically for C#).

Whenever you can write something in two or more different ways, use the one that is clearest and most stable against changes.

Re the secondary question: S3 should be placed inside the finally if it's concerned with cleanup. If it presupposes the success of the try block, it should be placed after the finally block. If your catch statement doesn't rethrow, I personally would interpret it to mean that you have succeeded and can proceed with normal operations. However, the whole 'Save the exception to re-throw later' thing confuses me. Generally, I'd advise against storing an exception for rethrowing outside the method. It's unusual and seems confusing to me. The fewer surprises your code contains, the easier it is to maintain (including yourself, three months later).

One case where you might not anticipate S1 throwing: if the thread is interrupted, the exception will automatically be rethrown at the end of the catch block anyway.

As Pontus says, a finally block indicates the intention that this code should always be run whatever happens. That's clearer than catching everything and then continuing.

finally block is used to do resource clean up either exception happens or not, finally block will both be called, so it's ideal place to do cleanup work.