What happens if a finally block throws an exception?

Quick (and rather obvious) snippet to save "original exception" (thrown in try block) and sacrifice "finally exception" (thrown in finally block), in case original one is more important for you:

try
{
    throw new Exception("Original Exception");
}
finally
{
    try
    {
        throw new Exception("Finally Exception");
    }
    catch
    { }
}

When code above is executed, "Original Exception" propagates up the call stack, and "Finally Exception" is lost.

If there is an exception pending (when the try block has a finally but no catch), the new exception replaces that one.

If there is no exception pending, it works just as throwing an exception outside the finally block.

For questions like these I usually open up an empty console application project in Visual Studio and write a small sample program:

using System;

class Program
{
    static void Main(string[] args)
    {
        try
        {
            try
            {
                throw new Exception("exception thrown from try block");
            }
            catch (Exception ex)
            {
                Console.WriteLine("Inner catch block handling {0}.", ex.Message);
                throw;
            }
            finally
            {
                Console.WriteLine("Inner finally block");
                throw new Exception("exception thrown from finally block");
                Console.WriteLine("This line is never reached");
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine("Outer catch block handling {0}.", ex.Message);
        }
        finally
        {
            Console.WriteLine("Outer finally block");
        }
    }
}

When you run the program you will see the exact order in which catch and finally blocks are executed. Please note that code in the finally block after the exception is being thrown will not be executed (in fact, in this sample program Visual Studio will even warn you that it has detected unreachable code):

Inner catch block handling exception thrown from try block.
Inner finally block
Outer catch block handling exception thrown from finally block.
Outer finally block

Additional Remark

As Michael Damatov pointed out, an exception from the try block will be "eaten" if you don't handle it in an (inner) catch block. In fact, in the example above the re-thrown exception does not appear in the outer catch block. To make that even more clear look at the following slightly modified sample:

using System;

class Program
{
    static void Main(string[] args)
    {
        try
        {
            try
            {
                throw new Exception("exception thrown from try block");
            }
            finally
            {
                Console.WriteLine("Inner finally block");
                throw new Exception("exception thrown from finally block");
                Console.WriteLine("This line is never reached");
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine("Outer catch block handling {0}.", ex.Message);
        }
        finally
        {
            Console.WriteLine("Outer finally block");
        }
    }
}

As you can see from the output the inner exception is "lost" (i.e. ignored):

Inner finally block
Outer catch block handling exception thrown from finally block.
Outer finally block

If a finally block throws an exception what exactly happens ?

That exception propagates out and up, and will (can) be handled at a higher level.

Your finally block will not be completed beyond the point where the exception is thrown.

If the finally block was executing during the handling of an earlier exception then that first exception is lost.

C# 4 Language Specification § 8.9.5: If the finally block throws another exception, processing of the current exception is terminated.