How do I pause during execution, save state, and continue from same point later on?

What you want could be accomplished by a serializable state machine. Basically, you change your local variables into fields in a class and add a field that keeps the state – the position in the code of the original method. This class will be [Serializable] and it will have one method like MoveNext(), which does a piece of work and returns. When working, you call this method in a loop. When you want to stop, you wait until the current call finishes, break out of the loop and then serialize the state machine to the disk.

Based on the complexity of the original method and how often you do want to “checkpoint” (when the MoveNext() method returns and you can choose to continue or not), the state machine could be as simple as having just one state, or quite complicated.

The C# compiler does very similar transformation when it's compiling iterator blocks (and C# 5's async methods). But it's not meant for this purpose and it doesn't mark the generated class [Serializable], so I don't think you could use it. Although reading some articles about how this transformation is actually done might help you do the same yourself.