how to use try catch blocks in a value returning method?

It would be better to do it this way,

 public bool CheckFileType(string FileName)
 {
    bool result = false ;

    try
     {
      string Ext = Path.GetExtension(FileName);
      switch (Ext.ToLower())
      {
        case ".gif":                   
        case ".JPEG":                    
        case ".jpg":                  
        case ".png":                   
        case ".bmp":                   
            result = true;
            break;
       }

      }catch(Exception e)
      {
         // Log exception 
      }
      return result;
     }

There are plenty of ways that you can use exceptions in methods that return values:

Place your return statement outside the try-catch For example:

T returnValue = default(T);
try
{
    // My code
}
catch 
{
    // Exception handling code
}
return returnValue;

Put a return statement inside your catch

try
{
    // My code
}
catch 
{
    // Handle exception
    return default(T);
}

Throw an exception

You don't have to return a value, the method simply has to end (e.g. reach a return statement or a throw statement). Depending on the exception its not always valid to return a value.

You should think carefully about when and how to catch and handle exceptions:

1. What might fail?
2. Why / how can they fail?
3. What should I do when they fail?

In your case:

1. The only statement that can fail is string Ext = Path.GetExtension(FileName);, which according to the documentation can fail if FileName contains. (Note that GetExtension doesn't return null, even if FileName is null).
2. This might happen if the user supplied a string that contains these invalid characters.
3. If this happens, I guess that we should return false, to indicate that the path is not valid (however this depends on the application).

So I'd probably handle exceptions like this:

public bool CheckFileType(string FileName)
{
    string Ext;
    try
    {
        Ext = Path.GetExtension(FileName);
    }
    catch (ArgumentException ex)
    {
        return false;
    }
    // Switch statement
}

Note that we only catch the exception that we are expected (ArgumentException), and we only place the try statement around the statement that we expect the exception to be thrown from.

In fact its a good idea to avoid throwing and catching exceptions wherever possible - not only do they incur a performance penalty (which can cause serious problems if this method is called inside a loop), but you might inadvertently catch and handle an exception that you didn't anticipate, masking a more serious problem.

In this case we can avoid throwing the exception entirely by checking ourselves to see if FileName contains any invalid characters:

public bool CheckFileType(string FileName)
{
    if (FileName == null)
    {
        return false;
    }
    if (FileName.IndexOfAny(System.IO.Path.GetInvalidPathChars()) >= 0)
    {
        return false;
    }
    // Your original method goes here
}

As you're not actually testing the file type (only the extension of the filename), I'd first start by renaming the method. You can make an extension method to handle it:

public static bool HasImageExtension(this string fileName)
{
    try
    {
        if (fileName == null) return false;

        string[] validExtensions = new string[] { ".gif", ".jpg", ".jpeg", ".png", ".bmp" };

        string extension = Path.GetExtension(fileName);
        return validExtensions.Contains(extension);
    }
    // catch the specific exception thrown if there are 
    // invalid characters in the path
    catch (ArgumentException ex) 
    {
        // do whatever you need to do to handle 
        // the fact there are invalid chars
        throw; 
    }
}

Which you can then call, like so:

string fileName = "testFileName.jpg";
bool hasImageExtension = fileName.HasImageExtension();