Handling Web Service Timeouts While Performing Long-Running Database Tasks

The web service could run the queries in a threadpool and if the thread does not finish within, say 5 seconds (see Thread.Join()), the web service call returns the client a JobID instead of the result set which the client can then use to poll the server every few seconds to see if its query finished. When a thread finishes the results can be stored in a hash table until the client polls again.

One of the solutions we've used lately is to break apart huge database processes into separate parallel operations. Each operation is much smaller and designed to be as efficient as possible. The clients initiate the operations, spawn a few threads, and do whatever they can in parallel.

For example, we've broken apart some huge proceses into a series of steps like Start, Process 1 Chunk of Work, Finish, and Gather Report Data. The Process Work steps can run in parallel, but they can't start until the Start step completes. The Finish step needs to wait for all Process Work steps to complete.

Since the client is controlling the process, the client can report progress on exactly which step it is on.

I've encountered similiar problems in the past, and used one of the following 3 methods to resolve it:

1. Add all long running queries to a queue, and process these sequentially.
   In my case these were all complicated reports which where then emailed to the client, or which were stored in permanent 'temporary' tables, for viewing by clients after they had been notified.
2. We called a webservice using a JQuery call, which then called a javascript postback method when it was complete.
   This worked well when we didn't want to make the page load synchronise with what the web service was doing.
   However it did mean that that piece of functionality was not available until the long running process was complete.
3. The most complicated one.
   We popped up another window which displayed a progress bar, which also polled the server periodically.
   This used a session variable to determine how far along to show the progress bar.
   After the progress bar was initiated, a new thread was started which updated the same session variable periodically.
   Once the session variable value was set to 100, the popup window closed itself.
   The clients loved this method.

Anyway I hope one of those is of some help to you.