Task swallows the exception thrown

If you want to fire and forget, you can attach a continuation using ContinueWith. The current try-catch will not help you at all, as the exception is encapsulated inside the Task. If this is "fire and forget", than you can log the exception:

public static Task MonitorQueueEmptyTask(
                         string queueName, CancellationTokenSource tokenSource)
{
    return Task.Factory.StartNew<bool>(() =>
    {
        while (!QueueManager.IsQueueEmpty(queueName))
        {
            if (tokenSource.IsCancellationRequested)
            {                            
                break;
            }

            Thread.Sleep(5000);
            throw new Exception("Throwing an error!");
        };
    }, tokenSource.Token, TaskCreationOptions.LongRunning).ContinueWith(faultedTask =>
    {
        WriteExceptionToLog(faultedTask.Exception); 
    }, TaskContinuationOptions.OnlyOnFaulted); 
}

This, in turn, will not propagate the exception after it's thrown, but will provide a mechanism to log the error. If you want the exception to be properly handled, you can register to TaskScheduler.UnobservedTaskException. Additionally, you can set ThrowUnobservedTaskExceptions enabled="true" in your configuration if you want unhandled exceptions to terminate your application. ContinueWith will consider the exception "handled" once you look at the task.Exception property.

The exception is not swallowed; it's just that it doesn't occur on the thread that executes the try/catch block, but on the separate Task thread.

If you don't observe the task's result or exception, when the task is eventually garbage collected, it will throw an exception saying that the task was not observed. Unless you catch that by handling the TaskScheduler.UnobservedTaskException, it will crash the process.